Multi variate testing
Usability tests
Media surveys
Intercept studies
Multi-variate testing lets you randomize the order in which participants see prototypes, concepts, or designs in your study. If every participant sees Concept A and then sees Concept B right after, it’s easy for a bias to form across your responses. Multi-variate testing solves that by showing up to three variants in a random order to each participant.
Each multi-variate question can hold up to three variants, but you can add as many multi-variate questions as you need to a study. You can also choose to show each participant only one variant from each multi-variate question while ensuring an even distribution across all participants.
Note: Even distribution is dependent on the number of participants who complete your study. For example, if you invite ten participants but have three variants to test, one variant will have an additional response. If a participant begins a study but abandons it, Dscout will automatically prioritize showing their variant or their variant order to another participant.
Multi-variate settings
When adding a multi-variate question to your study, you can choose whether participants see all variants or if they only see one variant at random:
Randomization
Determines whether a given participant is shown one or all variants in the multi-variate question. Choose from the following options:
Show all (default): Participants will see all variants in this question presented in a random order.
Show one: Participants will only see one variant in this question at random.
Warning: This setting cannot be changed post-launch.
You can also choose to randomize the questions within a variant itself. To access this setting, click the variant you want to randomize:
Randomize questions (variant-level only)
When enabled, participants will see all questions contained within this variant in a random order.
Note: Randomization applies to all questions within a single variant. If your questions rely on a specific prototype that is part of that variant, you may want to leave randomization disabled as we cannot ensure that question will be displayed before any others.
Warning: This setting cannot be changed post-launch.
Add a multi-variate question to a study
You’ll add multi-variate questions on the Usability test, Media survey, or Mission tab of your study. Once you’ve added a question, you can then add new variants (and their associated questions) within it.
Navigate to the Usability test/Media survey/Mission tab of your study.
Click + Add in the center of the builder or in the left sidebar.
Under More, select Multi-variate.
(If applicable) Select Show one to only display one variant to each participant at random. By default, Show all is selected. For more information, see Multi-variate settings.
Note: You cannot change this setting once you’ve launched your study, so make this decision carefully.
Click a variant to expand it, then click + Add question to add that variant’s relevant questions. Here, you can also enable randomization within this variant. For more information, see Multi-variate settings.
Note: Task questions with associated task URLs will have Close task URL enabled by default inside of variants. For more information, see Task questions.
By default, a multi-variate question begins with two variants, but you can test up to three in each question. Click + Add variant to add another variant.
You can rename your multi-variate question and variants to study-specific names by clicking on them. This will make analysis easier, helping you find insights faster.
Tip: You can drag and drop variants between multi-variate questions or drag and drop them into question groups. You can also drag and drop entire question groups into a multi-variate question as a new variant. Keep in mind that each question can only hold up to three variants.
Note: You can preview your multi-variate question just as you can the rest of your study. However, to test the randomization first-hand, you must do so using Test mode. Test mode is only available for usability tests and media surveys. For more information, see Test a usability test or Test a media survey.
Piping and skip logic in multi-variate questions
Because multi-variant questions are automatically randomized, there are some restrictions on how and when you can use piping and skip logic.
Piping logic
Piping logic lets you recall answers from previous questions as participants progress through a study. In multi-variate questions, you can use piping logic to:
Recall answers from questions that precede the multi-variate question.
Recall answers between two questions within the same multi-variate question (unless the questions within that variant are randomized).
Because your variants will all be presented in different orders to each participant, you can’t recall answers across variants. For more information, see Piping logic.
Skip logic
Skip logic lets you send participants to specific questions based on their answer choices. In multi-variate questions, you can use skip logic to:
Skip participants directly to a multi-variate question (not a specific variant).
Skip participants to the end of a variant, moving them on to the next question outside of the multi-variate question. (Only available when Show one variant is enabled.)
Skip participants from one question to another within the same variant. (Only available if Randomization is disabled.)
If questions within a variant are randomized, skip logic is disabled for that variant. For more information, see Skip logic.
Analyze a multi-variate question
You can analyze questions in a multi-variate block just as you normally would that individual question type. However, from the Questions view, you’ll see these questions grouped together on the Responses tab for easy access:
Analyze variants by participant
By default, variants and their questions are displayed in the order they appear in the builder. However, you can view everything in the order it was presented to each participant by navigating to individual responses from the Participants view:
Variant tags
Dscout automatically creates tags to help you easily filter between participants who saw specific variants or who saw variants in a specific order. To access these, click the Filter icon in the top right corner of the Responses tab. Then, select Tags. You'll see tags for each variant nested beneath the name of your multi-variate question: