Machine Learning Approaches for Interactive Verification
Yu-Cheng Chou andHsuan-Tien Lin
Department of Computer Science & Information Engineering
National Taiwan University
May 14, 2014 (PAKDD)
Breast Cancer Screening
http://en.wikipedia.org/wiki/File:Mammo_breast_cancer.jpg
• input: X-ray images
• output: healthy (left) or breast cancer (right)
• unbalanced: many healthy (negative), few cancerous (positive)
• learning a good model: important
—part of KDDCup 2008 task toeliminate false positive:
ask human experts toverify(confirm) all positive predictions
What If Too Many Positive Predictions?
input
positive predictions human
experts output verified instances
if human experts cannot handleall positiveinstances frommodel
• hire more human experts (but money?)
• random sampling (but false positive?)
another possibility:‘learn’ a verification assistant (verifier)
Learner versus Verifier
input instances
verifier learner
human
experts output verified instances label query
label model
verification query
two stagessimilarly require human (labeling):
learningandverification
—save human efforts by combining the two?
Motivation: One-Dimensional Separable Data
m instances on a line
• approach 1: binary searchfor learning, then doverification
• approach 2: greedily doverificationaccording to current model
+ + + - - - -
init init
number of queries ‘wasted’ on negative instances
• approach 1: O(log m)
• approach 2: O(1)
—combiningmay help
Interactive Verification Problem
• instances: X = {x1, ...,xm}
• unknown labels: Y (xi) ∈ {−1, +1}
• in iteration t = 1, 2, · · · , T :
select adifferent instance qt from X to query Y (qt)
input
instances interactive
verifier
human experts
output verified instances query
feedback
goal: maximizePT
t=1[Y (qt) =1]
—verify as many positive instances as possible within T queries
Our Contribution
input
instances interactive
verifier
human experts
output verified instances query
feedback
an initiative to study interactive verification, which ...
• introduces a simpleframework for designing interactive verification approaches
• connects interactive verification with other related ML problems
• exploits the connection to designpromising approaches with superior experimental results
Simple Framework for Interactive Verification
input
instances interactive
verifier
human experts
output verified instances query
feedback
For t = 1, ..., T :
1 train a model by abase learnerwith all labeled data (qi,Y (qi))
—will considerlinear SVMand denote weights bywt
2 compute ascoring functionS(xi,wt)for each instance xi ∈ X
3 query a different instance withhighest score
differentscoring functions⇔ differentapproaches
Greedy
+ + + - - - -
init init
• greedy: the ‘most’ positive one is the most suspicious one S(xi,wt) =xi|wt
—verify greedily!
• same asapproach 2in motivating one-dimensional data
how to correctsampling biaswith greedy queries?
Random Then Greedy (RTG)
• random sampling for learningfirst
• greedy for verificationlater
• RTG: one-time switching with parameter
S(xi,wt) =
(random(), if t ≤ T xi|wt, otherwise
how to learn faster thanrandom sampling?
Uncertainty Sampling Then Greedy (USTG)
• active learning: similar tointeractive verificationwith different goals
input
instances active
learner
human experts
output model query
label
• USTG:active learning(byuncertainty sampling) first,greedy for verificationlater
S(xi,wt) =
( 1
|xi|wt|+1, if t ≤ T xi|wt, otherwise
how to do better thanone-time switching?
Another Related Problem: Contextual Bandit
input context
contextual bandit learner
environment
output total reward action
reward
input
instances interactive verifier
human experts
output verified instances query
feedback
interactive verification
= special contextual bandit +verified instances as rewards
Upper Confidence Bound (UCB)
interactive verification
= special contextual bandit +verified instances as rewards
• contextual bandit: balanceexploration(getting information) and exploitation(getting reward)
• interactive verification: balancelearningandverification
• UCB: borrow idea from a popular contextual bandit algorithm S(xi,wt) =xi|wt + α ·confidence on xi
• α: trade-off parameter betweenexploration(learning) and exploitation(verification)
four approaches to be studied: greedy, RTG, USTG, UCB
Comparison between the Four
learning intent verification intent switching
greedy none positiveness none
RTG random sampling positiveness one-time USTG active learning positiveness one-time UCB confidence term positiveness dynamic
greedy: special case of RTG ( = 0), USTG ( = 0), UCB (α = 0)
Data Sets
data set number of instances number of positive instances percentage of positive instances
KDDCup2008 102294 623 0.6%
spambase 4601 1813 39.4%
a1a 1605 395 24.6%
cod-rna 59535 19845 33.3%
mushrooms 8124 3916 48.2%
w2a 3470 107 3%
—resampled with 1000 negative andP positive instances
will show
1 P
T
X
t=1
[Y (qt) =1]
under T = 100
Effect of
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55
ε
performance
USTG RTG
(a) KDDCup2008 with P = 100
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6
ε
performance
USTG RTG
(b) KDDCup2008 with P = 50
‘naive’greedy ( = 0) better thanRTGandUSTG, why?
Good Properties of Greedy
Case 1: positive instance selected
• successfulverification :-)
Case 2: negative instance selected
• ‘most unexpected’ negative instance
• usually helplearninga lot:-)
greedy approach happy ‘:-)’ either way
Artificial Data that Fails Greedy
−10 −8 −6 −4 −2 0 2 4 6 8 10
−10
−5 0 5 10
• twopositiveclusters and one bignegativecluster
• greedyignoresbottom cluster:negative instances selected doesn’t help learning
need to query ‘far-away’ (less confident) instances
—UCB to the rescue
Comparison between UCB and Greedy
P = 50 KDDCup2008 spambase a1a
UCB (α = 0.2) 0.5968 ± 0.0031 0.7306 ± 0.0020 0.3915 ± 0.0034 greedy 0.5868 ± 0.0040 0.7467 ± 0.0024 0.3883 ± 0.0034
comparison × 4
P = 50 cod-rna mushrooms w2a
UCB (α = 0.2) 0.7333 ± 0.0024 0.9776 ± 0.0007 0.6160 ± 0.0024 greedy 0.7249 ± 0.0027 0.9710 ± 0.0014 0.5944 ± 0.0030
comparison
UCBwins( ) often, across data sets and P
Conclusion
• formulated a novel problemof interactive verification
• connectedthe problem to active learning andcontextual bandit
• studieda simple solutiongreedy
• proposeda promising solutionUCBvia contextual bandit
• validatedthat greedy and UCB lead topromising performance
Self-Advertisement: TAAI 2014 in Taipei
2014 Conference on Technologies and Applications of Artificial Intelligence
TAAI 2014 Cal l f or Paper
Introduction
* Agents
* AIApplications
* AIArchitectures
* Automated Reasoning
* Cloud Computing in AI
* Cognitive Modeling
* ComputerGames
* ComputerGames
* ComputerSupported Collaborative Learning & Personalized Learning
* ComputerVision
* Data Mining
* Distributed AI
* Evolutionary Computation & Genetic
* Evolutionary Computation & Genetic Algorithms
* Fuzzy Systems
* Hybrid Systems
* Information Retrieval
* IntelligentE-Learning & Tutoring
* IntelligentEnvironments
* IntelligentHuman-ComputerInteraction
* Knowledge-Based Systems
* Knowledge Representation
* Logics in AI
* Machine Learning
* Mobile Intelligence
* Mobile Intelligence
* NaturalLanguage Processing
* Pattern Recognition
* Planning
* Probabilistic and Uncertain Reasoning
* Problem Solving and Search
* Robotics
* Semantic Web
* Semantic Web
* SocialComputing
* Speech Recognition and Synthesis
* Ubiquitous IntelligentApplications
* Web Intelligence General Co-Chairs
Program Co-chairs Hsuan-Tien Lin
(NationalTaiwan University) Hsing-Kuo Kenneth Pao Jane Yung-Jen Hsu (NationalTaiwan University) Yuh-Jye Lee
(NationalTaiwan University ofScience and Technology)
Key Dates
Submission welcomed! Thank you
Chou & Lin (NTU CSIE) Machine Learning Approaches for Interactive Verification 20/20