Measuring the Effectiveness of Epiphany With A/B Testing
We worked with Fortune 500 companies to test the speed, effectiveness & cost of Epiphany compared with established processes.
We did this through A/B testing, comparing four course designs created with humans only, humans + ChatGPT and humans + Epiphany.
Using the Scrap Learning Methodology, we assessed training effectiveness by having target learners rate both course versions on a 5-point scale for factors like role relevance, likelihood of skill application, design effectiveness, course content, engagement levels, and practical applicability.
A/B Testing Results
Comparing the speed, quality & cost of the training design process with and without Epiphany.
-87.5% time reduction to go from brief to design
50% increase in likelihood of on-the-job application
65% increase in design effectiveness
| Quality Metric | Existing methods | With Epiphany |
|---|---|---|
| Overall quality ratings | 3.21 | 4.42 |
| Role relevance | 3.75 | 4.00 |
| Application likelihood | 3.00 | 4.50 |
| Design effectiveness | 2.88 | 4.75 |
| Approach | Design Time |
|---|---|
| Traditional design | 8 hours over ~2 weeks |
| Current Epiphany design | 3 hours |
What Instructional Designers say
Epiphany helped me to create the course I always dreamed of creating, but never had the time for
Learners
[The Epiphany course] was much more user-friendly, and information was presented in a way that resonated more, and easier to apply to everyday work scenarios