Dscout supports two features which allow users to clean their data by reporting an issue and excluding submissions from their final analysis dataset in screeners, usability tests, media surveys, diary studies, and interview studies. Read on or click the links below for more information!
Overview
Researchers can flag and/or exclude submissions and entries that fall into these categories:
- Dishonest content
- Inappropriate content
- Low effort
- Instructions not followed
- Suspected gen AI use
- Duplicate account
- Duplicate entries
- Technical errors
- Off-topic entries
Below we've included some examples for each of the flag types:
- Dishonest content: inconsistent answers within a submission, inability to provide proof of ownership for missions about specific products or items.
- Inappropriate content: rude or disrespectful language + media.
- Low effort: nonsensical answers or responses that suggest minimal to no effort was given.
- Suspected gen AI use: media that contains suspected (or obvious) AI-generated content, written responses that you suspect may have been generated using AI.
- Duplicate account: finding two instances of the same person in your mission.
Reporting an issue
When reviewing submissions and interviews, you can report quality concerns regarding the data received.
- To report an issue in diary studies and interview studies: select an individual entry or interview > click on the flag icon in the upper right hand corner.
- To report an issue in screeners, usability tests, and media surveys: select an individual application or submission on the Responses tab > tap the three dots in the top right corner next to the participant’s name > select Report an issue.
After tapping to report an issue, you will be prompted to select the main reason for reporting the issue:
- Select Participant for end-user quality concerns (inappropriate, dishonest, low effort, etc.)
- Select Technical when you suspect or observed a technical issue which resulted in issues with the final submission or interview
After selecting the main reason, you’ll be required to select a reason and provide comments regarding the issue observed.
Our Support team reviews all flags and takes action as needed; any context you can provide about your flag in the comment field helps ensure our team takes the appropriate action. This may include following up with the participant to share feedback or communication regarding corrective actions; feel free to reach out to our Support team (support@dscout.com) if you ever have any questions or concerns!
Report an issue: Screeners
Report an issue: Diary studies
Report an issue: Interview studies
Report an issue: Usability tests and media surveys
Requesting refunds and backfilling data
When reporting an issue in usability tests and media surveys, you’ll see a few resolution options available to you for data cleaning, refund, and backfilling purposes. See screenshot and additional guidance below:
Exclude from my dataset
The entry will be excluded from your dataset, but will not be refunded or backfilled. You may reinclude data as needed when you select this option.
Request refund
Within 7 days of an entry’s submission date, you can optionally choose to request a refund for the participant’s incentive when reporting an issue to Dscout. When you select this option, the entry will be excluded from your dataset; please note that this cannot be undone, and will withhold the mission incentive from the participant. This is available only within 7 days of the participant submitting their entry to the mission, as incentives are automatically processed within 8 days of the submission date. At the top of the reporting menu, you will see the status of the participant’s payment alongside their payout date.
Request refund and recruit additional participant
Within 7 days of entry’s submission date, you may also choose to backfill the participant’s data as well as requesting a refund by selecting this option to maintain your original submission goal. If your mission is still launched, your mission openings will automatically increase by one; if your mission is closed, the system will automatically reopen, increase your openings by one, and automatically close after the new submission is received.
Excluding data
If you want to hide entries or submissions from your data set for analysis or data cleaning purposes, you can do so by using the exclude data feature. You may choose to use this feature in conjunction with the “report an issue” feature or on its own, depending on your reason for excluding the data. Excluded data will not appear in any of the Entry views (such as the Entries grid, Closed-ended Responses page, or Open-ended responses page) and will not appear in any exports from the mission.
- To exclude data in diary studies and interview studies: select an entry or interview > click on the exclude data icon in the upper right hand corner (next to the flag icon) > confirm that the entry should be excluded.
- To exclude data in usability tests or media surveys: select an individual submission (Responses tab) > tap the three dots in the top right corner next to the participant’s name > select Exclude entry from data set.
Excluding data: Diary studies
Excluding data: Interview studies
Excluding data: Usability tests and media surveys
Re-including data
Excluded data will not be deleted from the platform and researchers will be able to re-include any entries or interviews that may have been excluded accidentally. (Note: If you have requested a refund for a submission in a usability test or media survey, you will not be able to access the data or re-include it in your data set.)
- To re-include a diary entry: click the “Excluded entries” icon in the top right corner of the Entries view > select the entry in question > tap the “re-include” button in the top right corner
- To re-include an interview session: click the “Excluded sessions” icon in the top right corner of the Sessions view > select the interview in question > tap the “re-include” button in the top right corner
- To re-include non-refunded usability test or media survey entries: click the “View hidden entries” icon in the lefthand panel of your Responses tab
Re-including data will add it back to your main dataset, meaning the entry or interview will be included in counts, exports, and summary charts.