Boosting adoption of tutoring software

Client Ask

Explore ways to increase the adoption of CTAT, a tutor-writing software, in higher education settings.

Delivered

The design of a new website and tutorials for CTAT, and a built demo of these components.

Role UX designer

Team Grace Guo (researcher), Chris Feng (project manager), Siting Jin (developer)


Overview

Background on CTAT

The Cognitive Tutor Authoring Tools (CTAT) allow instructors to create intelligent tutors, which are powerful for their ability to provide contextual feedback as students work through problems. However, we found that potential users find the software difficult to understand, much less use.

Our Deliverables

We focused on helping users learn what CTAT can do. We designed web components that work together to teach users how to use CTAT to facilitate delivering problem sets and student feedback.


Target Users

Profile

  • Age: 30-50
  • Profession: Professor or instructional designer
  • Works in higher education & STEM domain

User Needs

  • To create challenging problem sets
  • To provide students with guidance and feedback

Our users are instructors in university-level settings. Providing fresh course material and feedback are both crucial to students' learning, yet both are immensely time consuming. For educators in STEM domains, CTAT could alleviate many of these issues, if they knew what it could do.

Faculty are busy...if they’re going to put in time to learn something, it better be worth it.
Director, Eberly Center for Teaching Excellence & Educational Innovation

Research

Key Findings

Through competitive analysis and interviews with professors, we learned about the problem-writing process and some useful features of other educational technologies. By comparison, CTAT is an extremely powerful tool but faces 3 major barriers to adoption:

  1. UI/UX: CTAT's interface is developer- (not user-) centered, resulting in frustrating and unintuitive user experiences.
  2. Troubleshooting: There is a lack of comprehensive documentation or help forums for troubleshooting.
  3. Onboarding: New users find CTAT overly complicated, rather than appreciating its advanced capabilities.

Design Goals

Of the 3 areas, we determined that only #2 and #3 were in scope for our time frame, as #1 would involve working with CTAT’s existing code base. We derived our goal to help instructors learn and adopt CTAT so they can more effectively help their students.

Prototypes

Low-Fidelity Prototypes

We identified tutorials as an entry point to engage with CTAT, hypothesizing that effective tutorials can increase interest. We based our prototypes on CTAT's existing tutorials (our control variable), asking users to follow the steps and what they retained afterwards. We tested 2 formats:

Format 1: Text and Image

Pros

  • Text provides clearer explanations and leads to better retention

Cons

  • Static images are hard to decipher without text
  • Explanations still feel long

Format 2: Video and Voiceover

Pros

  • Animation is easier to follow and increased task completion

Cons

  • Easy to miss explanations in the voiceover
  • Less understanding of features and what is happening

We found that text provided clearer explanations, which helped users retain specific parts better, but following a video made the task easier to complete.

Mid-Fidelity Prototypes

In the next rounds we worked to find a good balance between the two formats for better learning and retention.

During this process, I created interactive prototypes for testing, and iterated on the interactions and visual elements.

We found that initial transitions were too fast and jumpy. Users also wanted a way to move between steps.
In later iterations, we added navigation and a table of contents to give users a better sense of place and autonomy.
The tutorial made me want to know more. If it only uses a few things to make something decent, I want to know what else it can do!
Teaching Assistant at Carnegie Mellon University

Other Process

Website

We decided to rework CTAT’s website as a whole, because tutorials alone could not provide enough context about CTAT. I prototyped the web pages for testing, though not as extensively as our tutorials. Below are wireframes and older iterations.

Visual Components

As the team's designer, I also developed a new visual system for the site and tutorials, exploring component styles and layouts.

Typography & Icons

Callouts

Layout


Final Deliverables

Website

Our new website hosts a tutor gallery and a series of tutorials to teach new users about CTAT. I crafted the style and language of the site to be approachable and soothing. Below is a site map of our key pages.

Tutor Gallery

The gallery allows users to explore real, deployed tutors made with CTAT, demonstrating what they can do and serving as inspiration for novices.

Tutorials

The bulk of our work was focused here. The tutorials are designed as step-by-step guides to teach users about CTAT and the power of its features.


Learning Outcomes

  • Deriving design goals from open-ended user research
  • Balancing user needs with client's ask

Next Steps

Though users showed better learning and understanding of CTAT in our testing, we were not able to track the effectiveness of our design on CTAT adoption over time. Next steps might include:

  • Deploying and evaluating the long-term effect of the new website and its features on CTAT adoption. A good indication of tutorial effectiveness could be a positive correlation between views and tutors created.
  • Creating tutorials for CTAT's more advanced features. Making complex concepts learnable could encourage the creation of more diverse and effective tutors.