How to get user research insights on a product forum
What does nail polish and IBM have in common? Keep reading to find out!
Last month, we hosted a new(ish) type of event for one of our products on the IBM Community — our official forum for customers. We called it the IBM Community Innovation Jam. Our goal was to bring customers into our product-decision making process.
We started on the discussion forum for our data platform product, Cloud Pak for Data.
Community Innovation Jam Trailer
mediacenter.ibm.com
Tell me more, tell me more!
How did it work? We picked a time for everyone to get onto the Cloud Pak for Data Community discussion forum. I acted as the host of the event and posted prompts every 5 minutes related to different aspects of Cloud Pak for Data. We highlighted areas that design teams support and needed feedback on.
As a way to encourage participation, we labeled each post with a raffle ticket and told them that we’d enter them into a raffle for an IBM swag box or gift card.
The posts were relatively simple. On the majority of posts, we gave a high-level overview of the feature and then asked an open-ended question about it.
We also experimented with a couple of easier-to-answer questions that were more multiple-choice or short answers.
In a world where we seem to have more online meetings and events than ever, this was a more casual way for our customers to engage with us that did not require additional time spent watching something.
The goal of the event was threefold:
- provide valuable content to the members of our community
- gather insights for our product teams
- gain new sign ups for our User Experience Program and for specific research activities
Providing valuable content
In this first event, I worked with the various design squads on Cloud Pak for Data to gather a variety of relevant content for our participants. Specifically, I wanted to highlight features actively in development. That way, participants were not only able to provide actionable feedback, but also see the latest features.
We featured the Cloud Pak for Data homepage, user management, deployments dashboard, service catalog, and more.
One participant found it so valuable that she created a summary document to share with the community!
Gathering insights
As designers, we always want to make sure we’re getting feedback from the right people. At this first event, I honestly didn’t know what to expect! To my delight, we ended up having 14 participants: 6 internal users and 8 external customers.
Even though we mostly just posted static screens, we were able to get some helpful feedback! We had some nice pats on the shoulder as in the below example for the homepage.
Primarily though, we got constructive feedback like the one below where participants gave thoughtful responses on how to improve a certain feature.
Gaining new signups
On certain posts, where applicable, we linked to either a screener survey or a Calendly link where participants could sign up to give more detailed feedback on the feature.
At the end of the event, we offered extra raffle tickets for participants to sign up for the User Experience Program, what we call our customer panel. We didn’t get as many as we would like, but as we do more of these, I expect these numbers to grow.
All in all, I think we can safely say that we achieved those goals, but for the next time I’d like to exceed them!
The nail polish connection
My friend, Christine, started selling Color Street Nail Polish as a way to make some extra money to help pay for grad school. It’s like Avon, Pampered Chef, LipSense, Plexus, or one of those other type of multi-level marketing consultant businesses where distributors make money selling products to customers and recruiting new distributors.
It seems like a bunch of my Facebook friends started doing this type of thing. I get invites all the time to Facebook groups for this thing or that thing. Not sure if you’ve ever been invited to one of these types of events, but I always tended to ignore them.
But, Christine is one of my best friends. When she invited me to her first virtual “nail party,” I did not ignore the invite because I wanted to support her.
To my surprise, I had a lot of fun!
After going to a few of these virtual nail parties, I saw an opportunity to do something like this at IBM. I leveraged the basic framework of the virtual nail parties and applied it to IBM products.
(psst…if you like nail polish check out it out here! I’ve never had such long lasting, easy to put on nail polish)
Lessons learned for future forum engagements
As always, when you run something for the first time, there are going to be things you learn for next time!
While the event exceeded all our expectations, it definitely had areas for improvement. We also had some things that worked really well and even an unexpected outcome from the event.
Areas for improvement
1. Discussion posts easily got buried
Due to the default settings on the discussion forum, the posts that surface to the top are the ones that have had the most recent activity on them. It makes sense for normal forum uses, but for an event like this where we’re posting something every 5 minutes, it becomes hard to keep track of the new posts rather than just seeing the heavily commented older posts.
Actions to resolve: we can number the posts and/or utilize the forum’s tagging feature so that participants can filter just on the tag created for the Innovation Jam.
2. Balance between detailed responses with easy to answer responses
I tried to think of ways to create a variety of question types. Most of the prompts were around specific design features and getting feedback on them, but I tried to also mix in a few more easy to answer questions that were pseudo multiple choice. For instance, this question about challenges to adopt AI did not require a more detailed response. Most of the participants did include an explanation, but some did not.
Actions to resolve: Consider whether you need or want a more detailed response. Frame your questions so that they aren’t one-word answers or yes/no responses.
3. No good way to find out company
Through the tool we use for the Community forum, we can see the emails that the participants signed up with. The challenge there is that they might use a personal email rather than their work email and they’re not required to put in their company name to reduce the barrier to entry.
Actions to resolve: Consider making one of your prompts ask them for their company name or perhaps their LinkedIn profile. It’s highly probable that not everyone will respond or even feel comfortable responding, but it’s one way to attempt to find out their company.
4. Handling teammates jumping into the conversation
As this type of event gains more visibility, we need to think about how to set expectations with people at our company for responding to participants. Generally, we want to encourage peer-to-peer communication, but there may be times when the participants have a question that requires a response from a subject matter expert. There are also times when someone at IBM might want to ask a follow-up question to a participant or send them the documentation to help them learn more.
Actions to resolve: Consider reminding fellow teammates to keep a positive tone during the event and remind them that the main focus is to foster communication between community members.
What worked well
I definitely recommend doing the following for a similar event.
1. Scheduling posts
If you want to stay sane during an event like this, you want to not only pre-plan your posts, but also want to schedule them. Most forum tools should support this. Higher Logic, our vendor for the IBM Community, enables the scheduling of posts quite easily. When you draft a post, you have the option of publishing now or scheduling.
2. Emojis and visuals
Utilizing emojis in the subject line helped distinguish the event posts from the other forum-related posts. It made it very quick and easy for participants to find the posts for the event.
The posts themselves with visuals that the participants could respond to generated more engagement than posts without visuals.
3. Forum team approving first time posters
By default, the IBM Community forum site has a setting that first-time posters need to be approved. Since we invited people who might not have been members of the community, we needed to approve their posts ASAP. Since I was busy facilitating the conversation on the forum itself, having the IBM team who works on the community moderating and approving first-time posters was a lifesaver!
4. Post event Daily Digest
The IBM Community has a setting to send out a “Daily Digest” of activity in the forum to members. The Digest not only provided a nice summary of the event, but it also brought in some participants who did not engage during the event time itself.
Unexpected Outcome
You can’t anticipate everything when planning! One nice and unexpected outcome of this event was that one of the community members created a summary of what she’d learned during the event and shared it with the rest of the community. I loved seeing that and it proved that the event was valuable to at least one person!
Reflection
All in all, this was a crazy idea I had based on a Facebook group event I went to. I felt weird explaining to leadership that my idea came from a nail polish party, as I thought I’d get judged for that or something. However, rather than laughing off my idea, leadership got behind it with full support and helped me make this possible.
Moral of the story: Pay attention to the things you do outside of work. You never know where inspiration will strike you!
See all the posts from the event here, if you’re curious! And check out some awesome nail polish here if you’re into that kind of thing 🙂
Originally published on the UX Collective’s Bootcamp Medium Publication.
Rachel is a user experience researcher and strategist on the IBM Cloud and Cognitive Software Research Operations team.
The above article is personal and does not necessarily represent IBM’s positions, strategies or opinions.
