As noted, one goal of measuring eyetracking is to see whether these behavioral measures enable us to improve upon predictions of theory. This section reports whether using eyetracking data helps predict deception and uncover the underlying true states. The ability to detect private information in this way could eventually have many practical applications. And since private information often undermines efficiency, the ability to detect private information could be Pareto-improving in some settings.
Here, we ask how well receivers could predict the true state using only messages and lookup patterns (and how much they could earn by using those predictions). That is, we pretend we don’t know the true state for predictive purposes, forecast it from observables, then use knowledge of the true state to evaluate predictive accuracy. We focus only on b=1 and b=2 since truth-telling is so prominent when b=0.
For the dependent variable STATE j, from 1-5, we ran an ordered logit regression
[
≥]
=θ +∑
β ⋅ +β ⋅ +β ⋅ ⋅ +εsubjects. For other intervals, as predictive power (R2) falls the reliability across the two subsamples falls too, but the
where lookups are consolidated into two integer variables, ROWself and ROWother, which are the states corresponding to the own (or opponent) payoff row which has the longest total lookup time of all rows.
The coefficients β1b represents the information about the state contained in the message the coefficient, β2b measures the effects of the “most viewed row” of one’s own payoffs (i.e., the state number corresponding to the row that is viewed for the longest time), and β3b represents the effects of the “most viewed row” of the opponent’s payoffs. The θj are state-specific constants.
To evaluate how well these specifications could predict new data, out-of-sample
validation is used. Each observation is used with probability 2/3 to estimate the model, then the model forecasts on a holdout sample of the remaining 1/3 of the data. For each holdout
observation, the estimated logit probabilities are used to calculate the expected state, which is rounded to the nearest integer to make a precise single-state prediction. This partial estimation-prediction procedure is performed for 100 random samples of the data. Average βs and
(bootstrap) standard errors across the 100 resamplings are reported in Table 9.
The significance of β1b in Table 9 indicates that the messages are informative about the states (as analyses reported above established). A smaller message indicates a smaller true state, even though standard game theory predicts that little information should be transmitted by the message (β1b should be zero when b=2).
The lookup data are significantly correlated with states as well. The coefficients β2b, on the most-viewed own row variables, and the coefficients β3b, on the most-viewed other row variables, are all positive and significant. Thus, lookup data improve predictability even when controlling for the message. For example, if the message is 4, but the lookup data indicate the
coefficient signs are almost always the same in the two subsamples and magnitudes are typically reasonably close.
subject was looking most often at the payoffs in row corresponding to state 2, then the model could predict that the true state is 2, not 4. This is to be expected, since Table 6 indicates
subjects look at the payoff rows corresponding to the true state five times more than other rows.
However, note that this sort of prediction can only come from a setting in which attention is measured. In addition, if senders knew their eye movements were being used to infer the state, they could of course change their lookups and undermine the predictions.
The error rates in predicting states in the holdout sample are never greater than 40 percent.
(Keep in mind that the error rates in equilibrium would be 60 percent and 80 percent.) Most of the wrong predictions from the logit model (70 percent) miss the state by one. The model accuracy is also substantially better than the actual performance of the receiver subjects in our experiments: Subjects “missed” (chose A≠S) 58.5 percent of the time when b=1, and missed 77.9 percent for b=2.
An interesting calculation is how much these predictions could potentially add to the receiver payoffs (cf. “economic value” in Camerer et al., 2004). For biases b=1 and b=2, the average actual payoffs earned by receivers who faced eyetracked senders in the random sample were 87.5 and 80.9. If receivers had based their predictions on the models estimated in Table 8, and chose an action equal to the model predicted state (for the holdout sample), their expected payoffs would be 101.7 for b=1 and 98.0 for b=2. Since the maximum payoff possible is 110, this is a large economic value of about 60 percent of the increment between actual and maximum payoffs.28 In fact, these payoffs are already close to what subjects actually earn when b=0 and
28 For b=1, economic value = (101.7-87.5)/(110-87.5) = 63%. For b=2, economic value = (98-80.9)/(110-80.9) = 59%. Analogous out-of-sample prediction results for the display bias-partner design are reported in Table S9.
Results are weaker than that of the hidden bias-stranger design, having a modest economic value of 44 and 24 percent.
there is no bias (100.85 in Table 3).29 These economic value statistics suggest that it could be possible to almost erase the cost to receivers of not knowing the true state just by looking at attention along with messages.
An important caveat to these analyses is that we do not know what would happen if the senders knew that their pupil dilation and lookups were being used to predict the true state.
Senders would try to signal-jam by looking at the payoffs corresponding to their message more often (a kind of faked sincerity), but it is possible that excessive pupil dilation or more detailed lookup patterns could distinguish such signal-jamming. Putting senders under time pressure might also make it difficult for them use such a deliberately misleading strategy. In any case, such experiments are natural follow-ups and could be easily done.
V. Conclusion
This paper reports experiments on sender-receiver games with an incentive gap between senders and receivers, such as managers or security analysts painting a rosy picture about a firm’s earnings prospects. Senders observe a state S, an integer 1-5, and choose a message M.
Receivers observe M (but not S) and choose an action A. The sender prefers that the receiver choose an action A=S+b, which is b units higher than the true state, where b=0 (truth-telling is optimal), or b=1 or b=2. The bias number b is the size of the incentive gap. Receivers know the payoff structure, so they should be suspicious of inflated messages M.
Our experimental results show “overcommunication”—messages are more informative of the state than they should be, in equilibrium. This result is consistent with a level-k model of communication anchored at level-0 truth-telling. To explore the cognitive foundations of
29 Such gains in the hidden bias-stranger design are not surprising since subjects are forced to look at the payoff table to discover the bias parameter, and they focus disproportionally on the “true state” row along the way.
overcommunication, eyetracking was used to record what payoffs the sender subjects are looking at, and how widely their pupils dilate (expand) when they send message.30
The lookup data show that senders look disproportionally at the payoffs corresponding to the true state. They do not appear to be thinking strategically enough by putting themselves “in other’s shoes,” looking and choice are roughly consistent with a cognitive hierarchy specified by the level-k model, starting from truth-telling.
Senders’ pupils also dilate when they send deceptive messages (M≠S), and dilate more when the deception |M-S| is larger in magnitude. In a simpler pilot design that is prone to memory and repeated game effects (the display bias-partner design), these behavioral results are also present. Together, these data are consistent with the underlying assumptions of the level-k model, and that figuring out how much to deceive another player is cognitively difficult. Gneezy (2005) and Sjaak Hurkens and Kartik (2008) found that changing the known costs to others from deception lowers deception by subjects, suggesting that guilt plays a role in limiting deception.
Complementing this finding, we find that guilt does not appear to be the sole driver of overcommunication, because senders who look at receiver payoffs more often are also more deceptive. In fact, Santiago Sánchez-Pagés and Marc Vorsatz (2007) show that
overcommunication is caused by the tension between normative social behavior and incentives for lying.
Furthermore, combining sender messages and lookup patterns, one can predict the true state and lower the miss rate of subjects by one half. Those predictions increase receiver payoffs
30 The sender-receiver paradigm also expands the quality of research on lie-detection in general: Deception in these games is spontaneous and voluntary (most studies use instructed lying); and both players have a clear and
measurable financial incentive to deceive and to detect deception (most studies lack one or both types of incentives).
up to 16-21 percent, which is an economic value of more than half of the maximum increase above what subjects actually earn in the experiment.
There are many directions for future research.
Within this paradigm, eyetracking receivers would be useful for establishing their degree of strategic sophistication in making inferences from messages. More generally, economic theories often talk vaguely about the costs of decision making or difficulty of tradeoffs. Pupil dilation gives us one way to start measuring these costs.
Many economic models also specify a cognitive algorithm that maps acquired
information into choices (e.g, dynamic programming applications which require looking ahead).
The idea of allocating attention has itself gotten attention in economics (Della Vigna, 2008) and in macroeconomic studies of “rational inattention” (e.g., Christopher Sims, 2006). In both cases, measuring attention directly through (now video-based) eyetracking could improve tests of theories which make predictions about both attention and choice, and how they interact, as in previous mouse-tracking studies, such as Costa-Gomes et al. (2001), Johnson et al. (2002), and Costa-Gomes and Crawford (2006). Given the novelty of using these two methods in studying games, the results should be considered exploratory and simply show that such studies can be done and can yield surprises (e.g., the predictive power of lookups and pupil dilation for inferring private state information).
In the realm of deception, two obvious questions for future research are: Are there substantial individual differences in the capacity or willingness to deceive others for a benefit?
And, can experience teach people to be better at deception, and at detecting deception? Both questions are important for extrapolating these results to domains in which there is self-selection and possibly large effects of experience (e.g., used-car sales or politics). In other domains of
economic interest, the combination of eyetracking and pupil dilation could be applied to study any situation in which the search for information and cognitive difficulty are both useful to measure, such as “directed cognition” (Xavier Gabaix et al., 2006), perceptions of advertising and resulting purchase, and attention to trading screens with multiple markets (e.g., with possible arbitrage relationships).
References
Complaint in Securities and Exchange Commission V. Henry M. Blodget, 03 Cv 2947 (Whp) (S.D.N.Y.), Securities and Exchange Commission Litigation Release No. 18115, April 23, 2003. Washington, DC: Securities and Exchange Commission, 2003.
Order against Henry M. Blodget, Securities and Exchange Commission Administrative Proceedings, File No.3-11322, October 31, 2003. Washington, DC: Securities and Exchange Commission, 2003.
Second Amended Complaint in Securities and Exchange Commission V. Richard A.
Causey, Jeffrey K. Skilling and Kenneth L. Lay, Civil Action No. H-04-0284 (Harmon) (S.D. Tx.), Securities and Exchange Commission Litigation Release No. 18776, July 8, 2004. Washington, DC: Securities and Exchange Commission, 2004.
United States District Court Final Judgement on Securities and Exchange Commission V.
Henry M. Blodget 03 Civ. 2947 (Whp) (S.D.N.Y.), Securities and Exchange
Commission Litigation Release No. 18115, Washington, DC: Securities and Exchange Commission, 2003.
Aboyoun, Darren C. and Dabbs, James N. "The Hess Pupil Dilation Findings: Sex or Novelty?" Social Behavior and Personality, 1998, 26(4), pp. 415-19.
Beatty, Jackson. "Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources." Psychological Bulletin, 1982, 91(2), pp. 276-92.
Bernheim, Douglas. The psychology and neurobiology of judgment and decision making. In P.
Glimcher, C. Camerer, E. Fehr and R. Poldrack (Eds.), Handbook of Neuroeconomics, Elsevier, 2008.
Berrien, F. K. and Huntington, G. H. "An Exploratory Study of Pupillary Responses During
Deception." Journal of Experimental Psychology, 1943, 32(5), pp. 443-49.
Bhatt, Meghana and Camerer, Colin F. "Self-Referential Thinking and Equilibrium as States of Mind in Games: fMRI Evidence." Games and Economic Behavior, 2005, 52(2), pp. 424-59.
Blume, Andreas; DeJong, Douglas V.; Kim, Yong-Gwan and Sprinkle, Geoffrey B.
"Experimental Evidence on the Evolution of Meaning of Messages in Sender-Receiver Games." American Economic Review, 1998, 88(5), pp. 1323-40.
Blume, Andreas; DeJong, Douglas V.; Kim, Yong-Gwan and Sprinkle, Geoffrey B.
"Evolution of Communication with Partial Common Interest." Games and Economic Behavior, 2001, 37(1), pp. 79-120.
Bradley, M. T. and Janisse, Michel P. "Accuracy Demonstrations, Threat, and the Detection of Deception - Cardiovascular, Electrodermal, and Pupillary Measures." Psychophysiology, 1981, 18(3), pp. 307-15.
Bradley, M. T. and Janisse, Michel P. "Pupil Size and Lie Detection - the Effect of Certainty on Detection." Psychology, 1979, 16(4), pp. 33-39.
Brennan, Michael J. "How Did It Happen?" Economic Notes, 2004, 33(1), pp. 3-22.
Bull, R. and Shead, G. "Pupil-Dilation, Sex of Stimulus, and Age and Sex of Observer."
Perceptual and Motor Skills, 1979, 49(1), pp. 27-30.
Cai, Hongbin and Wang, Joseph T. "Overcommunication in Strategic Information Transmission Games." Games and Economic Behavior, 2006, 56(1), pp.7-36.
Camerer, Colin F.; Ho, Teck-Hua and Chong, Juin-Kuan. "A Cognitive Hierarchy Model of Games." Quarterly Journal of Economics, 2004, 119(3), pp. 861-98.
Camerer, Colin F.; Johnson, Eric J.; Rymon, Talia and Sen, Sankar. "Cognition and
Framing in Sequential Bargaining for Gains and Losses," K. G. Binmore, A. P. Kirman and P. Tani, Frontiers of Game Theory. Cambridge: MIT Press, 1993, 27-47.
Chapman, C. Richard; Oka, Shunichi; Bradshaw, David H.; Jacobson, Robert C. and Donaldson, Gary W. "Phasic Pupil Dilation Response to Noxious Stimulation in Normal Volunteers: Relationship to Brain Evoked Potentials and Pain Report." Psychophysiology, 1999, 36(1), pp. 44-52.
Chen, Ying. "Perturbed Communication Games with Honest Senders and Naive Receivers,"
Unpublished paper, 2007.
Costa-Gomes, Miguel A. and Crawford, Vincent P. "Cognition and Behavior in Two-Person Guessing Games: An Experimental Study." The American Economic Review, 2006, 96, pp.
1737-68.
Costa-Gomes, Miguel; Crawford, Vincent P. and Broseta, Bruno. "Cognition and Behavior in Normal-Form Games: An Experimental Study." Econometrica, 2001, 69(5), pp. 1193-235.
Crawford, Vincent P. "Lying for Strategic Advantage: Rational and Boundedly Rational Misrepresentation of Intentions." American Economic Review, 2003, 93(1), pp. 133-49.
Crawford, Vincent P. "Look-ups as the Windows of the Strategic Soul: Studying Cognition via Information Search in Game Experiments." Perspectives on the Future of Economics:
Positive and Normative Foundations, Volume 1, Handbooks of Economic Methodologies, ed. by Andrew Caplin and Andrew Schotter, Oxford University Press, 2008.
Crawford, Vincent P. and Sobel, Joel. "Strategic Information Transmission." Econometrica, 1982, 50(6), pp. 1431-51.
Dickhaut, John; McCabe, Kevin and Mukherji, Arijit. "An Experimental Study of Strategic
Information Transmission." Economic Theory, 1995, 6, pp. 389-403.
Della Vigna, Stefano. "Psychology and Economics: Evidence from the Field." Journal of Economic Literature, 2008, forthcoming.
Dionisio, Daphne P.; Granholm, Eric; Hillix, William A. and Perrine, William F.
"Differentiation of Deception Using Pupillary Responses as an Index of Cognitive Processing." Psychophysiology, 2001, 38(2), pp. 205-11.
Gabaix, Xavier; Laibson, David; Moloche, Guillermo and Weinberg, Stephen. "
Information Acquisition: Experimental Analysis of a Boundedly Rational Model."
American Economic Review, 2006, 96(4), pp. 1043-1068.
Gneezy, Uri. "Deception: The Role of Consequences." American Economic Review, 2005, 95(1), pp. 384-94.
Goldwater, B. C. "Psychological Significance of Pupillary Movements." Psychological Bulletin, 1972, 77(5), pp. 340-55.
Hall, Brian J. and Murphy, Kevin J. "The Trouble with Stock Options." Journal of Economic Perspectives, 2003, 17(3), pp. 49-70.
Heilveil, I. "Deception and Pupil Size." Journal of Clinical Psychology, 1976, 32(3), pp. 675-76.
Hicks, R. A.; Reaney, T. and Hill, L. "Effects of Pupil Size and Facial Angle on Preference for Photographs of a Young Woman." Perceptual and Motor Skills, 1967, 24(2), pp. 388-&.
Hurkens, Sjaak and Kartik, Navin. "Would I Lie to You? On Social Preferences and Lying Aversion." Experimental Econoimcs, 2008, forthcoming.
Jacob, Brian A. and Levitt, Steven D. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating." Quarterly Journal of Economics, 2003, 118(3), pp. 843-77.
Janisse, Michel P. "Pupil Size and Affect - Critical Review of Literature since 1960." Canadian Psychologist, 1973, 14(4), pp. 311-29.
Janisse, Michel P. and Bradley, M. T. "Deception, Information and the Pupillary Response."
Perceptual and Motor Skills, 1980, 50(3), pp. 748-50.
Johnson, Eric J.; Camerer, Colin; Sen, Sankar and Rymon, Talia. "Detecting Failures of Backward Induction: Monitoring Information Search in Sequential Bargaining." Journal of Economic Theory, 2002, 104(1), pp. 16-47.
Kahneman, Daniel and Beatty, Jackson. "Pupil Diameter and Load on Memory." Science, 1966, 154(3756), pp. 1583-85.
Kang, Min Jeong; Hsu, Ming; Krajbich, Ian M.; Loewenstein, George F.; McClure, Samuel M. ; Wang, Joseph Tao-yi and Camerer, Colin F. "The Wick in the Candle of Learning: Epistemic Curiosity Activates Reward Circuitry and Enhances Memory."
Psychological Science, 2008, forthcoming.
Kartik, Navin; Ottaviani, Macro and Squintani, Francesco. "Credulity, Lies, and Costly Talk." Journal of Economic Theory, 2007, 136 pp. (1), pp. 749-58.
Kartik, Navin. "Strategic Communication with Lying Costs." Review of Economic Studies, 2008, forthcoming.
Lin, Hsiou-wei and McNichols, Maureen F. "Underwriting Relationships, Analysts' Earnings Forecasts and Investment Recommendations." Journal of Accounting and Economics, 1998, 25(1), pp. 101-27.
Lubow, R. E. and Fein, Ofer. "Pupillary Size in Response to a Visual Guilty Knowledge Test:
New Technique for the Detection of Deception." Journal of Experimental Psychology-Applied, 1996, 2(2), pp. 164-77.
Michaely, Roni and Womack, Kent L. "Conflict of Interest and the Credibility of Underwriter Analyst Recommendations." Review of Financial Studies, 1999, 12(4), pp. 653-86.
Nietzsche, Friedrich. Human, All Too Human: A Book for Free Spirits, 2nd edition. Translated by R. J. Hollingdale, Cambridge University Press, 1878, 1996.
Nuñeza, Jennifer Maria; B.J. Caseyb; Tobias Egnerc; Todd Harea and Joy Hirsch.
"Intentional false responding shares neural substrates with response conflict and cognitive control." NeuroImage, 2005, 25(1), pp. 267-277.
Oka, Shunichi; Chapman, C. Richard and Jacobson, Robert C. "Phasic Pupil Dilation Response to Noxious Stimulation: Effects of Conduction Distance. Sex, and Age." Journal of Psychophysiology, 2000, 14(2), pp. 97-105.
Rosovsky, Henry and Hartley, Matthew. "Evaluation and the Academy: Are We Doing the Right Thing? Grade Inflation and Letters of Recommendation," Cambridge, MA: American Academy of Arts and Sciences, 2002.
Sánchez-Pagés, Santiago and Marc Vorsatz. “An Experimental Study of Truth-telling in a Sender-Receiver Game.” Games and Economic Behavior, 2007, 61(1), 86-112.
Selten, Reinhard. “Properties of a measure of predictive success.” Mathematical Social Sciences, 1991, 21, 153-167.
Sims, Christopher. “Rational Inattention: Beyond the Linear-Quadratic Case.” American Economic Review, 2006, 96, 158-163.
Table 1: Behavioral Predictions of the Level-k Model
Sender Message (condition on State) Receiver Action (condition on Message)
State 1 2 3 4 5 Message 1 2 3 4 5
b=0
L0/Eq Sender 1 2 3 4 5 L0/Eq Receiver 1 2 3 4 5
b=1
L0 Sender 1 2 3 4 5 L0 Receiver 1 2 3 4 5
L1 Sender 2 3 4 5 5 L1 Receiver 1 1 2 3 4
L2 Sender 3 4 5 5 5 L2 Receiver 1 1 1 2 4
Eq Sender 4 5 5 5 5 Eq Receiver 1 1 1 1 4
SOPH Sender 3 4 5 5 5 SOPH Receiver 1 2 2 2 4
b=2
L0 Sender 1 2 3 4 5 L0 Receiver 1 2 3 4 5
L1 Sender 3 4 5 5 5 L1 Receiver 1 1 1 2 4
L2 Sender 4 5 5 5 5 L2 Receiver 1 1 1 1 4
Eq Sender 5 5 5 5 5 Eq Receiver 1 1 1 1 3
SOPH Sender 5 5 5 5 5 SOPH Receiver 2 2 2 2 3
Note: L0 senders are truthful and L0 receivers best respond to L0 senders by following the message. L1 senders best respond to L0 receivers, while L1 receivers best respond to L1 senders, and so on. Note that when b=2, due to discreteness both L2 and Eq(=L3) senders best respond to L1 receivers.
Table 2: Information Transmission: Correlations between states S, messages M, and actions A
Note: In the hidden bias-stranger design, some senders’ eye movements were recorded (“eyetracked”) and others were not (“open box”). This comparison provides a useful test of whether obtrusively tracking a subject’s eye fixations affects their behavior.
Note: a Payoffs are not exactly the same due to the random noise added and certain groups excluded.
Table 4: Level-k Classification Results
Table 5: Average Sender Lookup Times (in seconds) Across Game Parameters
Bias
Table 6: Average Lookup Time per Row Depending on the State
Table 7: Individual Lookup Linear Measure Scores for Various Level-k Types
Type Subject ID L1 L2 L3/EQ
Note: Highest lookups scores underlined. Lookup scores if choice
classifications correspond to lookups boldfaced. Note that they almost always coincide for L1 and L2 types.
*, ** and *** denotes p<0.05, p<0.01, p<0.0001 for signed rank sum test using both own and other cells for each state, each bias, and each subject (of that type) with total lookup time > 1sec.
Table 8: Pupil Size Regressions for 400 msec Intervals
Y PUPILi
-1.2~
-0.8sec
-0.8~
-0.4sec
-0.4~
0.0sec
0.0~
0.4sec
0.4~
0.8sec
constant α 107.27 108.03 106.19 109.56 108.67
(2.81) (2.55) (2.57) (2.05) (2.16)
LIE_SIZE * BIASb β10 2.83 2.36 3.07 5.35** 5.57*
interactions (1.85) (2.23) (2.46) (1.76) (2.19)
β11 -1.02 -0.46 -0.36 2.16^ 2.64*
(1.26) (1.31) (1.28) (1.21) (1.15)
β12 2.06* 1.52^ 1.47* 1.83* 2.00**
(0.86) (0.79) (0.75) (0.75) (0.74)
N 414 415 414 415 414
χ2 323.86 235.43 194.40 258.49 352.49
R2 0.291 0.299 0.263 0.365 0.438
Note: Robust standard error in parentheses; t-Test p-values lower than ^10 percent, *5 percent, ** 1 percent, and *** 0.1 percent. (Dummies for biases, states, individual subjects and individual learning trends are included in the regression, but results are omitted.)
Table 9: Predicting True States (Resampling 100 times) (s. e. in parentheses)
X Hidden Bias-Stranger
MESSAGE * BIAS = 1 β11 0.46** (0.12)
MESSAGE * BIAS = 2 β12 0.42** (0.09)
ROW self * BIAS=1 β21 1.07** (0.24)
ROW self * BIAS=2 β22 1.72** (0.20)
ROW other * BIAS=1 β31 1.27** (0.22)
ROW other * BIAS=2 β32 0.44** (0.15)
total observations N a 357
N used in estimation 238.3
N used in estimation 238.3