In spring 2020, we tried to move one of our most hands-on workshops onto Zoom. The first attempt went badly; the second went much better. Here are three lessons for anyone moving in-person learning online.
If we listed what we (think we) do well as an organization, designing interactive learning experiences and workshops would be near the top. We love paying attention to the details: how people are welcomed into the space, the use of design and hands-on materials, changing body positions, mixing up open-ended and focused thinking, the creative use of case studies and more. So when COVID-19 restrictions on gathering arrived, we tried to recreate this, as much as possible, on Zoom.
We picked our impact measurement workshop, the Measurement Master Class, which we usually delivered as part of our Make Good course for social entrepreneurs. Normally, 15 to 20 participants sit in groups of four around tables with large interactive sheets full of graphics, text and illustrations. While there is an overall arc to the workshop, it is not linear. Participants have pens and sticky notes, play a game, discuss in small and large groups, and generally have a fun and meaningful experience. They leave with a better understanding of measurement, some practical tools, and what they need to start measuring their project’s impact.
Our plan was to stretch Zoom’s interactivity to its limits. We turned the large sheets into slides on a shared screen, planned to use annotation tools so people could fill in and draw on the slides, designed polls for the beginning and end, and used breakout rooms for all the discussion. We asked people to have their own sticky notes at hand. We worked with Learning Forte, a digital learning agency, to run the Zoom side. We found enough participants for two tests. We practiced. We prepared. Look out, digital learning, RootedGood is coming! (Or so we thought…)
Test 1: A trip to the dentist would have been more enjoyable
It started quite well, apart from the awkward introductions. With no clear way to set an order (apart from “Ben, please tell us your name…”) there is the inevitable silence, then three people speak at once, and then all are too polite to speak next.
Then the polling didn't work. We had a few co-hosts managing the workshop, and one of us closed the poll before others had voted. Then we couldn't see the results. Never mind: we ran the poll again and chatted about the results.
The opening content went fine, and we reached the first breakout room. We had carefully selected the groups and looked forward to hearing them discuss. Unfortunately, the breakout rooms did not work. Participants found themselves:
Kicked off the Zoom call altogether
In a breakout room on their own
In a breakout room with another screen with their name on it (this one caused some existential issues that might require therapy!)
Unable to leave the main room
We moved on, tried again at the second planned breakout, and it failed again. We had to cut all the remaining breakout rooms, and with them most of the discussion. The group was too large for everyone to discuss together, so we presenters had to talk much more than planned. So much for conversation.
Lesson 1: You need a backup plan
We had no backup plan. The interactive, discussion-based teaching we value quickly turned into a webinar! The shock. The horror. In trying to fix the problem, we lost time, rushed some sections, and some content felt clunky in the new format. A backup plan for the breakout rooms would have prevented much of this: the digital equivalent of having the caterer's number on speed dial in case lunch goes wrong.
The workshop ended. We thanked the participants (and apologized again). The call finished with that awkward moment where everyone says goodbye, then stares at the screen while looking for the “leave” button.
Lesson 2: Expect technical problems. At some point, they will happen
We probably apologized too much. We acted as if any technical problem was a complete failure on our part. Our expectations were unrealistic.
An analogy might help. I used to live and work in London (the UK one, not the one in Ontario). Most people who arrive for a meeting in central London use public transport to get there. Public transport is incredible in theory, tolerable in practice, and a nightmare on occasion. If you turn up late and mention a delayed train or broken-down bus, there is no condemnation, just knowing and sympathetic replies. It just happens.
Test 2: Confidence restored, but questions remain
We were tempted to cancel the second test, but didn’t. We investigated the Zoom problems and were (somewhat) confident the technology would work better. It did. The polling, the annotation, the breakout rooms: they all worked! We revised the content a little, didn't feel too rushed, and got really good feedback.
We measured this with a Net Promoter Score (NPS), which asks how likely people are to recommend something, on a scale from -100 to 100. After the first test, the NPS was -15. Not great. After the second, it was 82, a big jump almost entirely due to the technology working smoothly.
Clearly, the second test was much improved and people enjoyed it. But did it work as well as we hoped?
Lesson 3: Moving online takes a new workshop, not just new technology
We knew it would be a challenge, but we didn't appreciate how different a Zoom call is from a meeting room or coffee shop. Not everything that can happen in a room can happen on a call, and the reverse is true too. There are documented reasons why you feel different after a video call than after a similar conversation in person (BBC Worklife).
The possibilities and limits are different, and so are the learning styles and approaches needed. The workshop length needs to change. Content comes across in a more linear way on slides, so it needs to be designed that way. Participants will be more distracted and harder to engage, but likely more relaxed, with lower expectations.
We started by asking how to take our existing workshop, content and teaching approach, and put it online. We have now redefined the challenge: how do we design an online workshop to share what we know about impact measurement, and what content and teaching approach does that require?
Our main takeaway is that in the rush to “go digital,” we naively assumed technology would be neutral: that if we were creative enough, the same content and methods would get the same results. But as Marshall McLuhan famously said, “the medium is the message.” The context and method are not neutral. We all need to keep experimenting, and learn together how to learn in new environments.