| Applies to | Usability tests and media surveys |
Multi-variate testing lets you randomize the order in which participants see prototypes, concepts, or designs in your mission. If every participant sees Concept A and then sees Concept B right after, it’s easy for a bias to form across your responses. Multi-variate testing solves that by showing up to three variants in a random order to each participant.
Each multi-variate question can hold up to three variants, but you can add as many multi-variate questions as you need to a mission. You can also choose to show each participant only one variant from each multi-variate question while ensuring an even distribution across all participants.
Multi-variate settings
When adding a multi-variate question to your mission, you can choose whether participants see all variants or if they only see one variant at random:
| Setting | Description |
|---|---|
| Randomization |
Determines whether a given participant is shown one or all variants in the multi-variate question. Choose from the following options:
Warning: This setting cannot be changed post-launch.
|
You can also choose to randomize the questions within a variant itself. To access this setting, click the variant you want to randomize:
| Setting | Description |
|---|---|
| Randomize questions (variant-level only) |
When enabled, participants will see all questions contained within this variant in a random order.
Note: Randomization applies to all questions within a single variant. If your questions rely on a specific prototype that is part of that variant, you may want to leave randomization disabled as we cannot ensure that question will be displayed before any others.
Warning: This setting cannot be changed post-launch.
|
Add a multi-variate question to a mission
You’ll add multi-variate questions on the Usability test or Media survey tab of your mission. Once you’ve added a question, you can then add new variants (and their associated questions) within it.
To add a multi-variate question:
- Navigate to the Usability test or Media survey tab of your mission.
- Click + Add in the center of the builder or in the left sidebar.
- Under More, select Multi-variate.
-
(If applicable) Select Show one to only display one variant to each participant at random. By default, Show all is selected. For more information, see Multi-variate settings.
Note: You cannot change this setting once you’ve launched your mission, so make this decision carefully. -
Click a variant to expand it, then click + Add question to add that variant’s relevant questions. Here, you can also enable randomization within this variant. For more information, see Multi-variate settings.
Note: Task questions with associated task URLs will have Close task URL enabled by default inside of variants. For more information, see Task questions.
By default, a multi-variate question begins with two variants, but you can test up to three in each question. Click + Add variant to add another variant.
You can rename your multi-variate question and variants to mission-specific names by clicking on them. This will make analysis easier, helping you find insights faster.
Piping and skip logic in multi-variate questions
Because multi-variant questions are automatically randomized, there are some restrictions on how and when you can use piping and skip logic.
Piping logic
Piping logic lets you recall answers from previous questions as participants progress through a mission. In multi-variate questions, you can use piping logic to:
- Recall answers from questions that precede the multi-variate question.
- Recall answers between two questions within the same multi-variate question (unless the questions within that variant are randomized).
Because your variants will all be presented in different orders to each participant, you can’t recall answers across variants. For more information, see Piping logic.
Skip logic
Skip logic lets you send participants to specific questions based on their answer choices. In multi-variate questions, you can use skip logic to:
- Skip participants directly to a multi-variate question (not a specific variant).
- Skip participants to the end of a variant, moving them on to the next question outside of the multi-variate question. (Only available when Show one variant is enabled.)
- Skip participants from one question to another within the same variant. (Only available if Randomization is disabled.)
If questions within a variant are randomized, skip logic is disabled for that variant. For more information, see Skip logic.
Analyze a multi-variate question
You can analyze questions in a multi-variate block just as you normally would that individual question type. However, you’ll see these questions grouped together on the Responses tab for easy access.
Analyze variants by participant
By default, variants and their questions are displayed in the order they appear in the builder. However, you can view everything in the order it was presented to each participant by navigating to individual responses under Participants.
Variant tags
Dscout automatically creates tags to help you easily filter between participants who saw specific variants or who saw variants in a specific order. To access these, click the Filter icon in the top right corner of the Responses tab. Then, select Tags.