Online vs. in-person research? The recurring question many of us face every time we launch a new research initiative. There are obvious benefits and drawbacks to both, depending on objectives. Online will yield quick answers, with strong sample sizes, while in-person will deliver the most accurate in-situ read, with a more robust understanding of behavioral dynamics. And, of course, there are situations where both online and in-person methodologies are appropriate for a single study.
Pre-pandemic, the general consensus was to focus more on in-person methodologies for shelf-based testing. If there is a pack or planogram that needs to be tested, a very common and very reliable approach is to utilize in-person virtual reality testing. This is still the gold standard, but the effects of the global pandemic include a natural and necessary inclusion of more online methodologies.
So, what does an online shelf test need to deliver on to be as reliable as possible? Is “Online VR” an accurate testing approach?
It depends.
As you may suspect, the answer to this question depends on a few key necessary criteria. Online understanding of shelf behavior will never be as good as in-person, but there are a few key elements we need to get right in order to make it as reliable as possible.
Namely:
Viewing Angles
Traditional 2D online shelf tools tend to result in products being viewed that would not necessarily be noticed to the same extent in the real world. These methodologies typically rely on showing all (or a portion) of a shelf, top-to-bottom, for the respondent to shop. The respondent can equally notice products on the top, middle, or bottom shelves, giving every product an equal opportunity for success.
Unfortunately, the real world can be harsher. With a real shelf, the respondent starts on one end of the section, and works their way through the set. They unfortunately do not notice all products equally and are much more likely to notice products that are eye-to-waist height.
This is a problem that online VR can help solve. If done right, we can appropriately recreate real-world viewing angles, which more accurately depict true at-shelf behavior.
Fidelity
Not all environments are created equal. When conducting any online VR shelf testing, it is important to see the environment before field, and ask yourself if it is actually living up to its promise of recreating the environment. When a respondent enters the shopping exercise, you want them to be able to clearly make out the different products, and you want them to be viewing an accurate representation of what your team intended to be on the shelf (particularly for pack testing). You want respondents to be able to walk up to the products to get a closer look, or even pick up the products prior to purchase.
These are all elements that need to be included for a successful online VR tool.
Familiar Navigation
Many online VR tools provide navigation via keyboard controls, similar to video games. Although this method of control can seem intuitive, it often results in unnatural paths generated within the environment, as well as a snails-pace navigation experience.
Instead of this navigation method, we prefer mouse-based movement, where all controls follow essentially the same mouse navigation as any popular online map tool. This navigation method is natural for the vast majority of the population, primarily due to everyone’s familiarity with using such tools. When you get into an environment like this, you simply expect it to work in a similar fashion to your past experiences. This method also provides the added benefit of jumping to specific points of interest, skipping the slow hover to the desired location.
In short, mouse-based navigation is preferred.
Reliable Eye Tracking Proxies
It goes without saying that eye tracking from a webcam will never be able to compete with a professional physical eye tracker (at least not for the foreseeable future). Having said that, by using a combination of eye movements and key anchor points on one’s face, we can achieve a more reliable visual gaze result than we would if we were just focusing on eye movement via a webcam.
For this reason, we want to make sure that any webcam visual tracking utilizes as many available data points as possible.
Real-Time Behavioral Analysis
And finally, visual tracking in an online shopping exercise is really most valuable if it is dynamic in nature. By that, I mean that the tracking should take place while the respondent is actually experiencing the environment.
This seems obvious but, in fact, many online shelf tools simply conduct a short ‘viewing exercise’ after the actual shopping exercise, in order to understand what respondents are noticing on the shelf. This means that the results being captured are not live results from when the respondents were actually in the environment.
So, yes, ensuring that the online solution offers dynamic visual tracking is important.
Not perfect, but more reliable than a basic 2D shelf
Of course, online shelf testing will never be as good as someone actually shopping a real shelf. But online provides significant benefits, including speed and sample size. The key to having confidence in these online results, though, is making sure we are doing everything we can to make the behavioral experience as similar as possible to the real thing.
At Explorer, we have also conducted research on research around the comparability of in-person vs. 2D online shelf vs. online VR. We would be happy to share these results. Please contact us if you are interested.
Mike Moussallem is the President of Explorer Research. He has more than 15 years of experience in quantitative and qualitative research, shopper marketing, and category management. He has held positions at TNS, Kraft, PepsiCo, and Nielsen. Prior to joining Explorer, he led the Retail and Shopper Insights practice and team for TNS in Canada. Before TNS, Moussallem also led the Shopper Insights team for Kraft Canada, spanning across all sectors and customers.
