Assignment 0 (due 8:59 pm, May 9)

In this assignment, you will
  • Submit the CORE Certificate of Completion to CrowdMark
  • Submit the project topic, team name and the list of team members to cs449.spring18@gmail.com

1. The TCPS 2 Tutorial Course on Research Ethics (CORE)

In this assignment you first need to complete the Course on Research Ethics (CORE), also known as the TCPS2 tutorial. This training is mandatory for all researchers who intend to engage in research with human participants.

From the Welcome Page, click on the Log-In link. On this page you can click on the registration link. Fill in the required fields and register with your uwaterloo email. A confirmation email will be sent to the email address that you provide. If you do not see it in your inbox, please check your spam or junk mail folder. Click on the activation link only once to activate your account or copy and paste the complete activation url supplied into the url window of your browser.

Once your account is activated you can proceed to the log in page and begin the tutorial. It can take up to 3 hours take to complete CORE, depending on how many examples and activities you explore. Once you begin, you can go through the modules at your own pace (your progress points are automatically saved) logging out and logging in again to resume your session. If you experience any difficulties, refer to TCPS 2: CORE Frequently Asked Questions.

You need to finish the online course and submit the certificate of completion with your name on it.

2. Finalized project teams and project topics

Form teams of (ideally) 4 people. If you are in CS649, you should form teams with other CS649 students. The title of the email should include ''CS 449'' or ''CS 649'' and the name of your team. In the email please provide:
- a list of your team members and their uwaterloo emails;
- the name of your team;
- the project topic you chose from the provided list.

Assignment 1 (due 8:59 pm, May 16)

Submit to CrowdMark. Use the provided LaTex template or Word template to write the assignment.

CS 449: 2-3 pages + appendix
CS 649: 3-4 pages + appendix

In this assignment you will make an initial presentation of your project. The assignment should include the following four sections:

1. Description of the project (What will you do? Why you?)
Describe the high-level problem that you are trying to solve, and observations that led you to identify this problem. How can your project address this problem? Take this as an opportunity to describe what motivates you to work on this specific project topic, and what potential value your solution can bring. That will naturally lead you to the next section.
Example: In the AirBnB video we watched in class: Joe Gebbia is talking about how they come up with the idea, that it was solving problems of finding place to stay during the conference, and how different they want this experience to be from the regular hotel stay.

2. Goals and Hypotheses (How will it help?)
List the specific goals of your application and hypotheses you will be testing. What problems/obstacles do you want to solve? What information about the user do you need to have in order to do so? What do you hypothesize about this information? Briefly justify your hypotheses. If you use information from external sources, make sure you provide citation/references.
Example: In the AirBnB video we watched in class: They realized that they need to help users to form trust, both for hosts and for guests. But how do you form this trust? Should there be communication prior to arrival? What should be communicated, would providing additional information help? If so, what kind of information and how much should people say?

3. Target users & Participants for the Studies (Who is it for?)
An important part of identifying your market segment is to understand who your target users are. Describe in words the 3-5 personas of your future application. What patterns of users' behaviour, goals and motivations does each persona represent? Provide screenshots of completed persona templates and empathy maps in the Appendix. Describe the participants (e.g., demographics, etc) that you will work with for your user studies. You will need to work with 3-5 users for each study, and these users should be representative of your target user groups.

4. Plan for the Exploratory User Study
The exploratory user study should include at least interviews (other methods are optional). The exploratory study allows you to test some of the hypotheses you described in the sections above, and to gather information you identified as important for your project. In this section, describe what you will do for the exploratory studies and details about the methodologies you plan to use. Describe what you are planning to achieve through the study, and how this information can be used for your project. Provide a list of interview questions you are planning to ask in the Appendix. Note that interviews should not be longer than 15 min per session. In addition, you will most likely see some new information you did not anticipate; when planning your study, avoid leading questions (see lectures) and leave space for free-form feedback from your participants.

Marking breakdown:
(All subsections must be present in the assignment)
Description of the project: 20%
Goals & Hypothesis: 20%
Target user groups & users for the studies: 25%
Plan for the exploratory user study: 25%
Quality of the presentation (Structure; Language; Visualisation): 10%
Total: 100%

Assignment 2 (due 8:59 pm, June 13)

Assign one person from your team to submit the group writeup (cc' everyone in the team) to cs449.spring18@gmail.com. Use the provided LaTex template or Word template to write the assignment.

CS 449: 2-3 pages + appendix
CS 649: 3-4 pages + appendix

Please, include the name of your team in a title of your assignment. Include in the appendix copies of all your consent forms to this assignment.

1. Exploratory study results
Report on your findings from your exploratory study, i.e., interviews and observations. In class, we have discussed various ways to analyze qualitative data: first, we use for that purpose affinity diagrams as a tool to help us identify common themes in our data, second, we use work models. In this section, you will
  • name and describe themes you have identified using an affinity diagram. Included (in the appendix) a picture of the affinity diagram you have created with your team. The picture should be readable, so if needed submit several.
  • create and describe at least cultural and flow work models. Name and describe what themes and breakdowns you have identified when working on work models. Include (in the appendix) a picture of each *consolidated* (combining all your participants) work model created with your team. The pictures should be readable.
  • conclude by discussing all your findings with respect to your original hypotheses (i.e., your original assumptions about the target users: what did you learn? Did you find information supporting your hypotheses? Was there anything unexpected in your results? Which of the identified problems will you focus on? What could be reasons for these problems? What in your data can provide insight to address these problems?
2. Design ideas
After formulating the results of your exploratory study and identifying specific problems you want to focus on, it is time to think about how we can solve these problems with design. In this section, you will focus on the following:
  • when creating design solutions, you first need to brainstorm possible high-level ideas of how a challenge can be approached. Describe here these ideas (goals) you have chosen for your design solutions. Describe how, in your opinion, they will address the challenge and why.
  • when design solutions are chosen, you need to visualize them. To visualize design ideas and the workflow, we have discussed user stories (storyboarding), sketches, wireframes (which includes UI and user flows). Describe your user stories / storyboard, sketches, wireframes (UI and user flows); include pictures of these in the appendix.
    Note: At this stage you only need to submit a rough layout (static) of an application you are designing. These sketches should not be paper prototypes yet.
3. Describe paper prototype evaluation study
How do we know that the solution we have designed would actually solve the challenge and suit our users' needs? We first test and evaluate the ideas in low-fidelity prototypes. In this section you need to:
  • describe the goals for your evaluation studies (what exact features/aspects of your design you want to test)
  • formulate and justify hypotheses for your evaluation studies (e.g., why you expect specific features to be successful)
  • describe the procedure for your paper prototypes evaluation studies (including verbal script of instructions).
Note: You do not need to include results from the paper prototype evaluation study.

ONLY FOR CS 649 STUDENTS:

Graduate students are required to have an additional section - literature review, relevant to the challenges you want to focus on and your design solutions. Please review some academic literature on behavioural, design, and/or social aspects related to the challenges and your solutions. Your assignment then should include 4 sections:
1. Exploratory study results
2. Design ideas
3. Related literature
4. Plan for prototype test and evaluation


Marking breakdown (449):
Exploratory study results 30%
Design ideas 36%
Test and Evaluation studies plan 27%
Quality of the presentation (Structure; Language; Visualisation) 7%

Marking breakdown (649):
Exploratory study results 26%
Design ideas 32%
Related literature review 15%
Test and Evaluation studies plan 22%
Quality of the presentation (Structure; Language; Visualisation) 5%

Presentation 1 (June 18, June 20)

Submit poster in PDF format to cs449.spring18@gmail.com .

Your talk and your poster should cover the work you have done so far. This includes: the initial broad problem you started with, the process of narrowing down your focus through desk research (and results of desk research), target user group(s) and personas, goals and methods for exploratory studies, results of exploratory studies, initial design ideas. In addition to the poster you need to have your paper prototypes ready for demonstration. Your talk should be 4-5 min long.

The mark for the poster presentation will include assessment of your talk (clarity, structure, included work process, timing), visual presentation (clarity, structure, included work process, relation to the talk), quality of paper prototypes, individual participation (required evaluation of your classmates` presentations).

Poster boards are 4'x5'. You can print a poster at Media.Doc

Here are some good examples of posters:
https://uist.acm.org/posterexamplegallery/
http://colinpurrington.com/tips/poster-design#templates

Here are some powerpoint and latex templates for posters (you can find more just by googling):
powerpoint: https://www.genigraphics.com/templates
latex: https://www.overleaf.com/gallery/tagged/poster#.WsTZNGYZNR0

In general, posters are used to support your conversation with people who are visiting your posters. Do not cram it with too much text, equations or graphs. Think about how you want to present your poster, and add just enough information on the poster to help you say various things.

Assignment 3 (due 8:59 pm, July 11)

Submit to cs449.spring18@gmail.com. Use the provided LaTex template or Word template to write the assignment.

CS 449: 2-3 pages + appendix
CS 649: 3-4 pages + appendix

Please, include the name of your team in a title of your assignment.
Attach copies of all your consent forms to this assignment.

1. Results of paper prototype evaluation
- Describe the paper prototypes that have been tested. Include in the appendix all tested screens and interactions, with short notes in the caption. - First reflect on results in relation to the study goals you have previously identified for your paper prototype evaluation. Did you meet all your goals? If no, what is your opinion on why that happened? Did your study design reflect your goals well and why?
- Then report on discovered themes and detailed results of your paper prototype evaluation study. How do your results correspond to your hypotheses? What trends/themes did you find? What specifically did you learn about your design in relation to each theme? You need to report both on positive results (what did you find to be good design decisions) and negative results (identified problem areas in your design). Give your analysis on why identified design problems may occur, what causes them.


2. Design progression
- Use this section to describe and discuss changes that should be made to your initial design (used in the tested paper prototype) to address prototype evaluation results. Explain in detail why you expect each specific suggested change to be beneficial compared to the original design.
- Optionally, you can support your design progression ideas with wireframes. Wireframes are important for visualization and documentation purposes. With high level of precision and details, they should reflect such aspects of your design as structure of content on your screens and intended user flow through screens. Thus, creating wireframes is a valuable step on your way from ideation to high fidelity product implementation.

3. Plan for high fidelity prototype evaluation
- First describe your goals for high fidelity prototype evaluation. What type of information do you expect to get and why do you need it? What questions do you want to answer through this evaluation?
- Describe your plan for heuristic evaluation. What set heuristics are you planning to use and why (in relation to your goals)? Provide a set of scenario based tasks you are planning to use for the evaluation and briefly discuss your reasoning for chosen scenarios. Note: heuristic evaluation is to be conducted with an assigned group and does not require consent forms since it includes only the class members.
- Describe your plan for cognitive walkthrough evaluation. Include a description of the study environment and verbal script of the instructions for your participants. Provide set of scenario based tasks you are planning to use for the evaluation. If it is different from the set for heuristic evaluation, discuss your reasoning. Note: consent forms are required.

If you are planning to use any additional methods, provide a detailed plan, including a description of the study environment and verbal script of the instructions for your participants.

Marking breakdown (449 and 649):
Results of paper prototype evaluation 35%
Design progression 36%
Plan for high fidelity prototype evaluation 24%
Quality of the presentation (Structure; Language; Visualisation) 5%

Presentation 2 + Demo Video (July 25)

In this assignment, you will
  • Submit poster in PDF format to cs449.spring18@gmail.com.
  • Include in the email a link to an uploaded YouTube video.

Video Demo requirements:

3 min video demonstrating your high-fi prototype. Think of it as a video presentation of your final product. You should have supporting comments in the video to explain what is being shown. You can also include any other aspects of work you have done, if you would like to and if time on the video allows you (for example, your paper prototype to show the progression of the design).

Final Poster Presentation requirements:

4.5 min (sharp), you should tell the story of your project, from the beginning to the high-fidelity prototype. Don't forget about introduction - what you are doing and why. You must include description and results of your exploratory study, evaluation of paper prototype and high fidelity prototype, as well as reflection on design changes you have made throughout the project. If you find any models or graphs helpful to illustrate your story, it is always good to have graphical representations. However, remember, that any work model is not a result itself, it is a method to analyze data, so it can serve only supporting purposes. You should have functioning high fidelity prototype ready to show. You may want to bring your paper prototype as well - while not required, it would nicely support your design story. Remember about aesthetics and readability of your poster - use proper font sizes, spacing, content organization, etc.

Final Report (due 8:59 pm, July 27)

Submit to cs449.spring18@gmail.com. Use the provided LaTex template or Word template to write the final report.

CS 449: 8-12 pages + appendix
CS 649: 12-16 pages + appendix

Please, include the name of your team in a title of your final paper.
Attach copies of all not yet submitted consent forms to your final submission as a separate file .

Your final paper should include all work you have done for your project throughout the course. As you might notice from the marking breakdown, it is important for us to see that you have addressed our feedback throughout the course.

While split into sections, final paper should still tell one coherent story of your project development.

Description of the project
Describe the general market segment you are targeting, potential contributions you see as important for the area, what potential value you are bringing to the area, what problems / empty niches you originally identified and how your project addressed them.

Goals of the project
What specifically you are focusing on within your market segment and what exact purpose(s) do you see for your project.

Product anticipated users
In this section describe user groups you have identified for your product and include personas you have created.

Exploratory study
In this section you are expected to have the following:
- description of goals, hypotheses and focus you have formulated for your exploratory studies,
- description of your participants (number, average age, gender distribution, other demographic information important for your study),
- description of methods you have used including brief summary of your study designs: what themes you covered in your interview question, etc.
- report on exploratory study results: describe and discuss identified themes and detailed results, problems (breakdowns) you decided to focus on.

Initial design ideas
- Discuss how you decided to address identified problems and why.
- Describe what specific information from your exploratory results you used in your design.
- Include sketches and images of your paper prototypes (final versions) - you can include some images in your main text for illustrative purposes, but appendix should contain all images, clear, readable and in good quality.

Only CS 649 student:
Literature review relevant to the challenges you want to focus on and your design solutions.

Paper prototypes and evaluation
In this section you are expected to have the following:
- description of goals and hypotheses you have formulated for your paper prototype evaluation study,
- description of your participants (number, average age, gender distribution, other demographic information important for your study),
- set of scenario/goal based tasks you gave to your participants,
- report on paper prototype evaluation results: describe and discuss identified themes and detailed results, your analysis of possible reasons for identified problems in design.

Design progress
- Describe design adjustments you have identified as required based on paper prototype evaluation results, provide corresponding reasoning.
- Include wireframes you have created - you can include some images in your main text for illustrative purposes, but appendix should contain all images, clear, readable and in good quality.

High fidelity prototypes and evaluation
In this section you are expected to have the following:
- description of goals you have formulated for your high fidelity prototype evaluation study,
- description and results of heuristic evaluation:
--- description of your participants,
--- name used heuristics and reasoning for choosing them,
--- description of used scenarios and goal-based tasks,
--- report on results
- description and results of cognitive walkthrough:
--- description of your participants (number, average age, gender distribution, other demographic information important for your study),
--- description of used scenarios and goal-based tasks,
--- report on results
- report on consolidated required design changes identified in the studies.

Conclusion
Use this section to provide comparison of the initial project ideas and the final result, to reflect on design process you went through and to include you final thoughts on the project.

Marking breakdown (CS 449):
Description and goals of the project - 10%
Product anticipated users - 7%
Exploratory study - 13%
Initial design ideas - 8%
Paper prototypes and evaluation - 14%
Design progress - 8%
High fidelity prototypes and evaluation - 15%
Conclusion - 10%
Feedback integration - 10%
Quality of the presentation (Structure; Language; Visualisation) 5%

Marking breakdown (CS 649):
Description and goals of the project - 9%
Product anticipated users - 6%
Exploratory study - 12%
Initial design ideas - 7%
Design-relevant literature review - 7%
Paper prototypes and evaluation - 13%
Design progress - 7%
High fidelity prototypes and evaluation - 15%
Conclusion - 9%
Feedback integration - 10%
Quality of the presentation (Structure; Language; Visualisation) 5%