行政院國家科學委員會專題研究計畫 成果報告
決策學習模型的通用架構：尺度不變性與參數估計
研究成果報告(精簡版)
計 畫 類 別 ： 個別型 計 畫 編 號 ： NSC 98-2410-H-194-113- 執 行 期 間 ： 98 年 08 月 01 日至 99 年 07 月 31 日 執 行 單 位 ： 國立中正大學心理學系 計 畫 主 持 人 ： 鄭中平 計畫參與人員： 博士班研究生-兼任助理人員：陳淑萍 報 告 附 件 ： 出席國際會議研究心得報告及發表論文 處 理 方 式 ： 本計畫涉及專利或其他智慧財產權，1 年後可公開查詢中 華 民 國 99 年 10 月 31 日
行政院國家科學委員會專題研究計劃報告
決策學習模型的通用架構：
尺度不變性與參數估計
A general framework for decision-learning
models: Dimensional invariance and parameter
estimation
計畫編號：
NSC 98-2410-H-194 -113執行期限：98 年 8 月 1 日至 99 年 7 月 31 日
主持人：鄭中平
中正大學心理學系
摘要
愛荷華賭博作業廣泛用在顯示不同群體在決策歷程中的 差異，透過以量化的認知模型分析愛荷華賭博作業，由於模 型中的參數分別對應愛荷華賭博作業心理歷程的成分，可進 一步瞭解不同群體在決策歷程上的可能差異。除了常用的期 望價值學習模型外，尚有近十個量化認知模型，企圖模擬愛 荷華賭博作業的認知歷程。本研究首先建立一個通用的架構 納入這些決策學習模型。其次，本研究針對此通用架構發展 較具一般性的估計方式，以進行這些模型的參數估計。最後， 透過通用架構進行這些模型的比對，瞭解模型的尺度不變性。 關鍵詞：愛荷華賭博作業、認知模擬、期望價值理論Abstract
Iowa gambling task is a cognitive task designed for exploring the
possible decision making deficit. By modeling this task, parameters
of the expectancy-valence model may be correspondent to the
components of psychological processes underlying the Iowa gambling
task. Instead of often-cited expectancy-valence model, there are up
to 10 quantitative decision-learning models aim to simulate Iowa
gambling task. Firstly, the study proposed a general framework to
incorporate most decision-learning models. Secondly, the study
developed a comprehensive statistical routine for most
decision-learning models under the framework. Finally, by comparing
these models under the framework, we will explore the scale
Statement of the study
The study aims to develop a general framework to incorporate
several decision choice models simulating the decision-making
process of Iowa gambling task (IGT, Bechara, Damasio, Damasio, &
Anderson, 1994). By doing so, we can implement one comprehensive
parameter estimation procedure for several models. We will develop a
comprehensive routine for all models, rather than develop several
specific statistical routines for every single model. Simply put, the
first aim of the study is to develop a general framework for IGT. The
second aim of the study is to develop a routine for parameter
estimation of these models. Third, we will check scale invariance
properties under the general framework.
Iowa gambling task and Models
The Iowa gambling task was developed to simulate real-life
trial, the participant should choose a card from four decks. Decks
differ in their frequencies and values of gain and loss. Gains of some
decks are higher, but their mathematical expectancies are negative.
When the participant chooses a card, the gain and the loss will show.
The participants are asked to maximize their outcome.
The task has been widely used to examine possible
neuro-cognitive deficits in normal and clinical populations. IGT has
been found to be useful in differentiating young and healthy subjects
from other target groups such as elders (Wood, Busemeyer, Koling,
Cox & Davis, 2005), substance abusers (Busemeyer, Stout, & Finn,
2007; Stout, Busemeyer, Lin Grant & Bonson, 2004; Verdejo-Garcia
et al., 2007; Yechiam, Busemeyer, Stout & Bechara, 2005), patients
with Huntington’sand/orParkinson disease(Busemeyer & Stout, 2002), and patients with damage of orbital frontal cortex (Bechara et
Busemeyer and Stout (2002) proposed the expectancy-valence
learning model (EV model) to simulate cognitive process underling
IGT. EV model consists of three parts. In the first part, the participant
is supposed to calculate the valence in each trial, and then form the
expectancy valence of every deck described in the second part. The
last part described the participant’s choice is based on the expectancy
valence calculated during the trials. There are three parameters
corresponding individual differences in the three parts in EV model
including attention to the gain, attention to recent outcome and the
change of sensitivity of response choice to expectancy valence across
trials.
Parameters of the EV model may be interpreted in
correspondence to the components of psychological processes
underlying the task. Estimates of model parameters can then be used
example, Busemeyer & Stout (2002) found higher learning rate
parameter estimates for the Huntington group than that in healthy
group; Cheng, Sheu, and Yen (2009) found that male adolescents pay
more attention to gains than female adolescents. Among these,
Yechiam et al. (2005) analyzed IGT data obtained from 10 different
populations of participants based on the EV model, and mapped
parameter estimates derived from each of the groups in the space of
parameters.
Comparison of model parameter estimates on a map may reveal
differences of decision making processes of distinct groups. If the
differences of parameter estimates between groups are the main
concern of the empirical studies undertaken, before comparing them,
the adequacy of comparisons should be checked.
Instead of EV model, Ahn et al. (2008) proposed seven other
consider an alternative for original description of every part, so there
are eight IGT models including EV model. They evaluate eight
models based on BIC (Bayesian Information Criterion) and
generalization criterion method. According to the criteria they
suggested, EV model is not the best among eight models. They
concluded that switching to the other model in the future should be
considered. However, as we shall see, the conclusion should be
suspended because some models they compare are not scale invariant
and comparisons of models which are not scale invariant may be
problematic.
A General Framework for IGT Models
In the section, we proposed a general framework to incorporate
Ahn’s eight models. By doing so, we can propose a general estimation
routine and explore the scale invariance of eight models in a unified
also consists of three parts, as followed: ( 0) ( )I wWt lLt | | t t t v wW lL (1) , (1 (( , 1) 1) ) 1, , t k t k t k t k k E a E a (2) 4 1, , , 1 exp( c ) / exp( c ) t k t k t j j P t E t E
(3)Because expectancy and prospect utility functions are combined
into equation (1), it is a little more complicated. In equation (1), v ,t
t
W and L denote the utility, win and loss at trial t.t , w , l and
are four parameters. I x( ) is a function which is 1 if statement x
is true and is o if x is false. In equation (1), if we let l 1 w, 1
, then equation (1) will be the expectancy valence function in Ahn et al. (2008). If we let l w 1, then equation (1) will become to be prospect utility function.
In equation (2), Evk t, is the expectancy utility of deck k at trial
k. is the index variable to indicate whether deck k is chosen ork t, not at trial t. At trial t, if the participant chooses deck k, k t, 1,
otherwise, k t, 0. a and are two parameters. If 1, equation (2) will be the delta learning rule It will be the decay
reinforcement rule if 0.
Equation (3) describes the choice probability of deck k at trial
t+1 Pt1,k is proportional to the expectancy utility of deck k at trial t.
The expectancy utility is weighted by a function of trial t. and c are two parameters. If , then it will be trial-dependent choice.1c rule. If c0, it will be the trial-independent choice rule.
Parameter estimation of single subject
The second aim of the study is to develop a parameter estimation
procedure for the general framework. We propose to estimate
parameters of the general framework via maximum likelihood method
and implement the estimation procedure by SAS.
Because equation (2) is defined recursively, we can combine all
, , 4 1, , , 1 1 1 exp( (1 ) ) / exp( (1 ) ) t t j m k m m s m s t t c c t k k s s j s s s j s P t a a v t a a v
(4)In equation (4), the dependent variable is the deck the participant
chooses at some trial t. The independent variables are all payoffs
before the trial t. The framework can be seen as a special case of
nonlinear regression models. For the dependent variable are
categorical (the chosen deck), so the nonlinear regression may be
some kind of multinomial logistic regression models. Parameters of
multinomial logistic regression can be estimated with maximum
likelihood method. SAS package NLMIXED is used in the study.
Parameter estimation of group data
Buesmeyer and Stout (2002) proposed a two-staged strategy to
analyze group differences in parameters of EV model. First, estimates
of model parameters are obtained separately from fitting the EV
model by the maximum likelihood method with Nelder-Mead simplex
optimization (O’Neill, 1971). The values of these individual
parameter estimates then serve as raw data for comparing group
differences via standard parametric statistical procedures such as the
Student’st-test or analysis of variance.
The estimation procedure in the first stage is found to be
susceptive to its start values during optimization and the two-stage
procedure may lower the power to detect group difference (Cheng,
Sheu, & Yen, 2009; Wetzels, Vandekerckhove, Tuerlinckx, &
Wagenmakers, 2010). In order to overcome optimization problem and
improve the power, Wetzels et al. suggested a Bayesian approach.
Instead of Baysian approach, Cheng, Sheu, and Yen (2009)
reparameterzied the EV model to overcome the optimization problem
and suggested a mixed-effects model to improve the power.
its mixed-effect version. All parameters in the framework will be seen
as subject-specific random variables, and are assume to follow a
multivariate normal distribution. Again, SAS package NLMIXED is
used for its ability to handle mixed effect nonlinear regression.
Scale Invariance of the general framework
A model is scale invariant if we change the unit of scale of the
model, we can find another set of values of parameters which make
the same prediction. For a scale invariant model, if the value of
parameter will not affected by the scale, we call it scale free.
Otherwise, we call it scale dependent (Cudeck, 1989).
With the general framework for IGT models, we investigate scale
invariance issue in the third study.
Let 4 1 1 1 1 1 1 ( | ) exp( ) exp( ) t t c c k t k l t l j l t l l j l p t s t s
(1 ) t k l m l D k l t k l where s a a D 4 1 1 1 1 1 exp( ) exp( ) t t c c k l t l j l t l l j l t s t s
4 1 1 1 1 1 exp( ) exp( ) t t c c k l t l j l t l l j l t s t s
1 (pk t| ) by setting In summary, only models with trial-independent choice rule are
scale invariant. Since some of eight models are not scale invariant,
comparison based on BIC may be not proper. All parameters except
sensitivity parameter in the models with trial-independent choice rule
are scale independent. Comparison of sensitivity parameters across
studies with different currency should be made with caution.
Conclusion and discussion
In the previous work, we found that several decision learning
models share the same structure with EV model (Cheng & Sheu,
2008a,b). Besides the EV model, we found out that the
Rescorla-Wagner model for conditioning (Rescorla & Wagner, 1972),
1955) and a logistic regression model for avoidance learning (Gelman,
Goegebeur, Tuerlinckx & Van Mechelen, 2000) can also be
reformulated under the framework. Some properties can be easily
observed under the learning-choice framework. For example, if we
focus on the effect of trial t, we can observe that the logistic regression
model is the only model whose learning rate depend on trial t. Its
learning rate decreases as trials increase. And in EV model and
logistic regression model, the probability of choice is a function of not
only expectancy-valence but also trial t. That is, even if the
expectancy valences of all options are the same at trial t1 and trial t2,
EV model and logistic model will predict different choice probabilities.
But for Rescorla-Wagner and Bush and Mosteller’s models, the
predictions are the same for t1 and t2, they have no memory.
With cognitive modeling, we can understand the different
assuming the model is appropriate. Previous studies show the benefit
from taking mixed effect into account in cognitive modeling (e.g.,
Cheng, et, al, 2009; Rouder et. al., 2005). The study may provide
some insight about the differences of psychological process under IGT
References
Ahn, W.-Y., Busemeyer, J. R., Wagenmakers, E. J., & Stout, J. C.
(2008). Comparison of decision learning models using the
generalization criterion method. Cognitive Science, 32,
1376-1402.
Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994).
Insensitivity to future consequences following damage to human
prefrontal cortex. Cognition, 50, 7-15.
Busemeyer, J. R., Stout, J. C., (2002). A contribution of cognitive
decision models to clinical assessment: Decomposing
performance on the Bechara Gambling Task. Psychological
Assessment, 14, 252-262
Busemeyer, J. R., Stout, J. C., Finn, P. R. (2007). Using computational
models to help explain decision making: processes of substance
abusers. Invited Chapter to be published in Cognitive and
Press.
Bush, R. R. and Mosteller, F. (1955). Stochastic models for learning.
New York: Wiley, 1955.
Cheng, C.-P., & Sheu, C.-F. (2008a, September). Learning Models and
the Iowa Gambling Task. Paper presented at the 39th Meeting of
the European Mathematical Psychology Group. Graz, Austria.
Cheng, C.-P., & Sheu, C.-F. (2008b, October). On the Relationship
between Expectancy-Valence Model and Other Learning Models.
Paper presented at the Annual Meeting of Taiwan Psychology
Association, National Taiwan Normal University, Taipei, Taiwan.
Cheng. C.-P., Sheu, C.-F., & Yen, N.-S. (2009). A Mixed-Effects
Expectancy-Valence Model for the Iowa Gambling Task.
Behavior Research Methods, 41, 657-663.
Chiu, Y. C., Lin, C. H., Huang, J. T., Lin, S., Lee, P. L., & Hsieh, J.-C.
decision makers in the Iowa Gambling Task? Behavioral and
Brain Functions, 4, 13.
Cudeck, R. (1989). Analysis of correlation matrices using covariance
structure model. Psychological Bulletin, 105, 317-327.
Gelman, A., Y. Goegebeur, F. Tuerlinckx & Van Mechelen, I. (2000),
“DiagnosticChecks for Discrete Data Regression Models Using PosteriorPredictiveSimulations,”Journal of the Royal Statistical Society: Series C (Applied Statistics), 49, 2, 247 –268.
Krantz, D.H., Luce, R.D., Suppes, P., Tversky, A. (1971). Foundations
of Measurement, Academic Press, New York, NY., Vol. 1.
O’Neill, R. (1971). Algorithm AS 47: Function Minimization Using a
Simplex Procedure. Applied Statistics, 20, 338-345.
Rescorla, R. A. & Wagner, A. R., (1972). A theory of Pavlovian
conditioning: variations in the effectiveness of reinforcement and
Classical conditioning II: current research and theory, 64–99,
New York: Appleton-Century-Crofts.
Stout, J.C., Busemeyer, J.R., Lin, A., Grant, S.J. and Bonson, K.R.
(2004). Cognitive modeling analysis of the decision-making
processes used by cocaine abusers. Psychonomic Bulletin and
Review, 11, 742-747
Verdejo Garcia, A., Benbrook, A., Funderburk, F., David, P., Cadet, J.
L., & Bolla, K. I. (2007). The differential relationship between
cocaine use and marijuana use on decision-making performance
over repeat testing with the Iowa Gambling Task. Drug and
Alcohol Dependence, 90, 2-11.
Wetzels, R., Vandekerckhove, J., Tuerlinckx, F., & Wagenmakers,
E.-J. (2010). Bayesian parameter estimation in the expectancy
valence model of the Iowa gambling task. Journal of Mathematical Psychology, 54, 14-27.
Wood, S., Busemeyer, J. R., Koling, A., Cox, C. R., & Davis, H.
(2005). Older adults as adaptive decision makers: Evidence from
the Iowa gambling task. Psychology and Aging, 20, 220-225.
Yechiam, E., Busemeyer, J. R., Stout, J. C. and Bechara, A. (2005)
Using cognitive models to map relations between
neuropsychological disorders and human decision making
出國報告
本次計畫中，前往荷蘭阿姆斯特丹，參與美國數學心理學學會（SMP）與歐 洲數學心理學研究群（EMPG）聯合年會。美國數學心理學學會年會與歐洲數學 心理學研究群年會是最大的兩個數學心理學方面會議，此次聯合舉辦，參與人數 相當多。 會議總共舉行三天，會前一天尚有關於貝氏推論的工作坊。本次會議邀請 G. Logan 做主題演講，他回顧了他的注意力模型；另外還安排青年獎得主 S. Brown 分享研究成果與經驗，。除大會演講外，平行的口頭報告同時間有三場，議程相 當緊湊。 我個人的報告是考慮模擬愛荷華賭局作業八種模型的尺度不變性，發現其中 四種模型（包括常用的期望價值模型）不具尺度不變性，亦即，當改變模型中報 酬（payoff）的單位時，模型的統計評估（AIC）與預測會隨之不同，但加入額外 參 數 後 此 問 題 可 獲 解決 。 報 告 時 ， 發 展 愛荷 華 賭 局 作 業 認 知 模擬 的 Prof. Busemeyer 亦在場，他說明了當初發展期望價值模型時，原先考慮多一參數，但 由於諸多原因而沒加入。本次會議除發表論文外，亦於會前參加一天的貝氏統計 應用於認知模擬之工作坊，瞭解如何利用 winstep 結合 R 或 Matlab 之介面， 以貝氏統計方式分析認知作業。98 年度專題研究計畫研究成果彙整表
計畫主持人：鄭中平 計畫編號： 98-2410-H-194-113-計畫名稱：決策學習模型的通用架構：尺度不變性與參數估計 量化 成果項目 實際已達成 數（被接受 或已發表） 預期總達成 數(含實際已 達成數) 本計畫實 際貢獻百 分比 單位 備 註 （ 質 化 說 明：如 數 個 計 畫 共 同 成 果、成 果 列 為 該 期 刊 之 封 面 故 事 ... 等） 期刊論文 0 0 100% 研究報告/技術報告 1 1 100% 研討會論文 0 0 100% 篇 論文著作 專書 0 0 100% 申請中件數 0 0 100% 專利 已獲得件數 0 0 100% 件 件數 0 0 100% 件 技術移轉 權利金 0 0 100% 千元 碩士生 0 0 100% 博士生 1 0 100% 博士後研究員 0 0 100% 國內 參與計畫人力 （本國籍） 專任助理 0 0 100% 人次 期刊論文 0 1 0% 投稿中。 研究報告/技術報告 0 0 100% 研討會論文 2 2 100% 篇 論文著作 專書 0 0 100% 章/本 申請中件數 0 0 100% 專利 已獲得件數 0 0 100% 件 件數 0 0 100% 件 技術移轉 權利金 0 0 100% 千元 碩士生 0 0 100% 博士生 0 0 100% 博士後研究員 0 0 100% 國外 參與計畫人力 （外國籍） 專任助理 0 0 100% 人次其他成果
(
無法以量化表達之成 果如辦理學術活動、獲 得獎項、重要國際合 作、研究成果國際影響 力及其他協助產業技 術發展之具體效益事 項等，請以文字敘述填 列。) 無 成果項目 量化 名稱或內容性質簡述 測驗工具(含質性與量性) 0 課程/模組 0 電腦及網路系統或工具 0 教材 0 舉辦之活動/競賽 0 研討會/工作坊 0 電子報、網站 0 科 教 處 計 畫 加 填 項 目 計畫成果推廣之參與（閱聽）人數 0國科會補助專題研究計畫成果報告自評表
請就研究內容與原計畫相符程度、達成預期目標情況、研究成果之學術或應用價
值（簡要敘述成果所代表之意義、價值、影響或進一步發展之可能性）
、是否適
合在學術期刊發表或申請專利、主要發現或其他有關價值等，作一綜合評估。
1. 請就研究內容與原計畫相符程度、達成預期目標情況作一綜合評估
■達成目標
□未達成目標（請說明，以 100 字為限）
□實驗失敗
□因故實驗中斷
□其他原因
說明：
2. 研究成果在學術期刊發表或申請專利等情形：
論文：□已發表 □未發表之文稿 ■撰寫中 □無
專利：□已獲得 □申請中 ■無
技轉：□已技轉 □洽談中 ■無
其他：（以 100 字為限）
3. 請依學術成就、技術創新、社會影響等方面，評估研究成果之學術或應用價
值（簡要敘述成果所代表之意義、價值、影響或進一步發展之可能性）（以
500 字為限）
With cognitive modeling, we can understand the different psychological processes in widely used IGT among populations assuming the model is appropriate. Although the often-cited EV model is questioned by its author, how to compare EV model with other possible models is still open. The study may provide some insight about the properties of decision- learning models for IGT and evaluates these models in a more appropriate manner.
