Measuring question sensitivity in Fake Good Fake Bad experiments

In this prestentation we provide an overview of results of several Fake Good Fake Bad experiments. In such experiments respondents are asked to intentionally give either socially desirable answers (fake good) or socially undesirable answers (fake bad). Significant differences between these two groups are used as an indication that the questions indeed are sensitive and thus vulnerable to social desirability effects. Our first experiment, in a web survey (n=215) on alcohol consumption, showed that variance of the scores in the fake bad condition was systematically higher than in the fake good condition. Several possible explanations exist for this phenomenon. It may mean that respondents are less determined about what is socially undesirable than about what is socially desirable, it may mean that faking good is easier (to understand) than faking bad, or it may mean that respondents interpret the faking instructions differently. In order to increase comprehension of the faking task, in our second and third experiment, we tested the effects of visual and oral instructions. Only the oral instruction that preceded administration did yield lower variance in the fake bad group, but not for all questions. Another cause of high variance in fake scores could be the different social settings that respondents take into account while determining the socially desirable or undesirable position. Therefore, in the fourth experiment, we tested, in a web survey (n=224) on alcohol and drug use, the effects of steering respondents towards a social context by means of pictures. However, results showed no difference in respondent’s answers for the version with pictures that included social context (i.e., pictures of people consuming alcohol or drugs) versus a version that included no social context (i.e. pictures of alcoholic beverages or drugs). Finally, in a fifth experiment, we tested the possibility of allowing respondents to select multiple response options. The results showed that only a fraction of respondents selected more than one option, which may have been due to satisficing effects.