Turning user workflows into product impact and explainable AI techniques
role
Design Research Intern
timeframe
9 weeks
description
Wild Me’s Wildbooks use computer vision to help conservation researchers identify animals by matching images in a global database.
At the SEI, I worked on a project exploring explainable AI (XAI) techniques through a case study with Wild Me.
I collaborated with SEI researchers, engineers, and CMU HCII designers to create user analysis tools that improve explainability in Wildbooks.
highlights
search_insights
Provided customer trend insights on over 30 real customer use cases.
timeline
Customer use research directly inspired product impact initiatives for Wild Me.
context
Discovering unmet explainability needs
Imagine using the Wildbook AI system while looking at two photos of the same whale tail. The system tells you that these tails are the same, but the markings on the tail are obviously different.
Wouldn't you want to know why AI says two images match, when they clearly don’t?
Current XAI Techniques
The SEI team completed previous need-finding to understand what XAI looks like in practice. There are currently 2 explainability techniques in the Wildbook system, match scores and saliency maps.
Match Scores
Calculates similarity between the uploaded image and images in the embedding space.
Saliency Maps
Highlights the parts of the image the model referenced most when making its decision.
These techniques have many constraints.
flowchart
Both techniques have technical limitations in implementation.
support_agent
Wild Me’s team faces challenges troubleshooting techniques with researchers.
psychology_alt
Researchers find current XAI techniques hard to interpret.
mystery
Researchers prefer tools that align with pre-AI visual matching protocols.
my goal
Build a tool for researchers to analyze data in the Wildbook system.
research
Machine learning from scratch
Prior to this internship, I had no real idea how machine learning models even worked, let alone how to build tools to explain them, let alone how to explain over 20+ Wildbook models. I completed a literature review to catch up on what the team was exploring before I was added to the project.
How might we design a user-centered AI analysis tool?
From previous research, I discovered some design principles for creating human-centered AI tools.
input
Clearly communicate how the users’ input data is used.
interactive_space
Empower interaction with the model: allow users to play and make discoveries.
emoji_people
Respect users’ autonomy and decision-making abilities.
Gaining insights through hands-on experience and researcher protocols
edit_document
Researchers create their own training videos and protocols for their species’ Wildbook.
groups
Researchers work in teams when matching individuals.
I was able to play around on different Wildbooks to discover myself how to navigate Wildbooks. Additionally, I watched and read various training protocols created by researchers to get a sense of individual team use cases.
problem space & ideation
Shaping an analysis tool through collaborative ideation
Building an “analysis tool” was an extremely vague task, and the team allowed me to define what that meant by what I was most interested in exploring during my internship. After ideation with the team on various problem statements, I landed on the following statement.
problem
Researchers matching animals with WildMe need many ways to interpret model outputs to reduce time and cognitive effort used to make a match.
I then developed a how might we question to guide sketches of various prototype solutions.
How might we display human-understandable model outputs in an interface?
challenge
Identifying gaps during prototype iteration and feedback
As I was getting feedback on these prototype sketches from the project lead, I found that there were still a lot of unanswered questions that were not discovered during the team's previous need-finding. Researchers were really interested in matching animals… But why?
discovery
We knew how researchers used the system, but I needed to uncover why they used it.
research (pt.2)
Understanding Wildbook's role in research workflows
Taking a step back to the research stage, I performed open and axial coding on 37 research papers that cited the use of Wild Me to get a deeper sense of how researchers used Wildbooks to answer their research questions.
Wild Me was also able to use this data to support their product strategy initiatives, exploring Wildbook's impact on sand tiger sharks.
Researchers:
crowdsource
Use many different sources for data collection.
question_exchange
Answer questions across a variety of topics with one system.
analytics
Have pre-AI methods to perform analysis on matches.
preview
Use manual key point matching techniques to verify matches recommended by the AI system.
problem space definition (pt. 2)
Navigating complexity with design exercises
After drawing insights from the use case research, I was unsure how to move forward with what I found. I decided to use some design thinking exercises to draw out some additional insights and guide a path for the next steps in defining the design problem.
Understanding system impact
checklist
Researchers are juggling a lot of data and a lot of jobs.
I used the data to derive jobs that each of these researchers needed to complete to understand how the system helped researchers accomplished the tasks they needed to complete.
Gaining the bigger picture
add_photo_alternate
Researchers need to make sure the images input into the system are good quality.
I took the main job from my JTBD breakdown and performed abstraction laddering. I chose this method since a bigger picture look was missing from the initial need-finding.
Intervening earlier
cognition
With improved input monitoring, researchers get more accurate results to perform more accurate analysis— in their own ways.
Discussing these exercises with the project lead, we deduced that maybe what users really need is explainability support earlier in the process. There are already methods of analysis outside of the Wildbook system. Wildbooks need more support for logistical explainability to support workflows without disrupting current patterns.
new problem
How might we ensure efficient data collection that produces accurate model outputs?
ideating solutions (pt. 2)
Iterating on maps to focus the solution
online_prediction
insight
Prototypes were juggling too many solutions at once.
new_releases
improvement
Ground the solution in a persona and scenario.
To avoid starting from scratch with the end of my internship fast approaching, I started by utilizing my original map prototype ideas. I took my sketches and created a medium fidelity prototype for users to monitor their collection methods based on a photo's location metadata. After showing my prototypes to the project lead, we decided my prototypes were trying to juggle too many solutions to one problem.
Using personas and scenarios to streamline the design
troubleshoot
Imagining how a researcher troubleshoots issues identified the necessary features.
Creating a specific scenario and persona helped narrow down the amount of pieces I was trying to juggle in my solution. It helped me think about specific scenarios and how a researcher might solve them with my solution.
Persona
Sand Tiger Shark Researcher
Wildbook Use Case
Collect data from live-streaming cameras to spot match individuals.
Motivations
Get initial counts to determine the right population statistical model
Discover how human interactions affect the conservation of sharks
Scenario
Research Team A is completing research on sand tiger sharks, and has 4 live-streaming cameras to capture photos of sharks. While monitoring the feed, they realize there are often sharks in the background that are not close enough to the camera to be identified. They realize that their camera traps are not capturing as many sand tiger sharks as they could be, and their data is not best representing the population. The team now needs to see how well effective the cameras have been so far, in order to decide if they should relocate the cameras and where they should be moved to.
Iterating for scale
online_prediction
insight
Researchers collect hundreds of photos from one camera.
new_releases
improvement
Indicate the number of photos taken by a single camera.
Sharing this prototype with my project lead, we identified that this iteration fails to acknowledge the hundreds or even thousands of images a researcher may collect from one camera.
final designs
Data collection monitoring dashboard
Considering a way to give researchers the same visual overview while indicating a large photo set lead me to this final prototype.
feedback
Gaining feedback and aligning with product goals
no_photography
Camera traps were not the correct terminology for the solution I had proposed.
trending_down
Incorporating camera data would be resource heavy.
Presenting my solution to the Wild Me product team allowed me to get feedback on my designs that aligned with their product strategy.
discovery
An iteration that included team data would fit better with the current implementation of Wildbooks.
conclusion
Looking ahead
Due to the end of my internship, I wasn’t able to show this prototype to users or continue iterating.
Future improvements with more time and resources
restart_alt
More iterations
I would like to design another iteration of the prototype that focused on more team or collaborative matching processes.
account_circle
Limited time for user testing
I wish i could have shown this to users to see if this solution better met their needs and workflows. Because of the back and forth in the problem definition stage, it was difficult to schedule users for testing.
forum
Bridging communication gaps
The SEI team only interfaced with the Wild Me product team every other week. Something as simple as the word “camera trap” not being the right language could have been easily solved if I worked more closely with the Wild Me team.
retrospective
Exploring future directions for explainable AI research
Overall, I think this project has really interesting implications for future explainable AI research. Expanding explainability beyond just the code or algorithm piece would be something I am interested in exploring in the future.
Final takeaways
partner_exchange
Design Strategists vs UI Designers
Working in the academic research space, there are a lot more design strategists than UI designers. While both roles have their places, being a design strategist can leave the UI designers who pick up the principles a bit confused on how to implement these abstract solutions. If academic strategists tried implementing their principles, they might find some interesting insights more quickly than just focusing on strategy.
model_training
AI interfaces do not need to reinvent the wheel
In this project, I reused a lot of design components I saw in other non-AI interfaces. AI might be “new” and “upcoming” but don’t need “new” designs. Communicating model outputs does not have to be designed in a shiny, innovative way.
analytics
Understanding data-heavy systems
I was able to work with data-heavy systems which was an awesome opportunity. I had never really worked with systems that had so much data, and it was a challenge that pushed my design skills. It would be great to try it again in a different opportunity.
up next
how about another story?
Accessible Health for People with Spinal Cord Injury
arrow_right_alt