Machine Learning Approaches for Interactive Veriﬁcation

(1)

Machine Learning Approaches for Interactive Verification

Yu-Cheng Chou andHsuan-Tien Lin

Department of Computer Science & Information Engineering

National Taiwan University

May 14, 2014 (PAKDD)

(2)

Breast Cancer Screening

http://en.wikipedia.org/wiki/File:Mammo_breast_cancer.jpg

• input: X-ray images

• output: healthy (left) or breast cancer (right)

• unbalanced: many healthy (negative), few cancerous (positive)

• learning a good model: important

—part of KDDCup 2008 task toeliminate false positive:

ask human experts toverify(confirm) all positive predictions

(3)

What If Too Many Positive Predictions?

input

positive predictions human

experts output verified instances

if human experts cannot handleall positiveinstances frommodel

• hire more human experts (but money?)

• random sampling (but false positive?)

another possibility:‘learn’ a verification assistant (verifier)

(4)

Learner versus Verifier

input instances

verifier learner

human

experts output verified instances label query

label model

verification query

two stagessimilarly require human (labeling):

learningandverification

—save human efforts by combining the two?

(5)

Motivation: One-Dimensional Separable Data

m instances on a line

• approach 1: binary searchfor learning, then doverification

• approach 2: greedily doverificationaccording to current model

+ + + - - - -

init init

number of queries ‘wasted’ on negative instances

• approach 1: O(log m)

• approach 2: O(1)

—combiningmay help

(6)

Interactive Verification Problem

• instances: X = {x₁, ...,x_m}

• unknown labels: Y (x_i) ∈ {−1, +1}

• in iteration t = 1, 2, · · · , T :

select adifferent instance q_t from X to query Y (q_t)

input

instances interactive

verifier

human experts

output verified instances query

feedback

goal: maximizePT

t=1[Y (qt) =1]

—verify as many positive instances as possible within T queries

(7)

Our Contribution

input

verifier

human experts

feedback

an initiative to study interactive verification, which ...

• introduces a simpleframework for designing interactive verification approaches

• connects interactive verification with other related ML problems

• exploits the connection to designpromising approaches with superior experimental results

(8)

Simple Framework for Interactive Verification

input

verifier

human experts

feedback

For t = 1, ..., T :

1 train a model by abase learnerwith all labeled data (q_i,Y (q_i))

—will considerlinear SVMand denote weights byw_t

2 compute ascoring functionS(x_i,wt)for each instance x_i ∈ X

3 query a different instance withhighest score

differentscoring functions⇔ differentapproaches

(9)

Greedy

+ + + - - - -

init init

• greedy: the ‘most’ positive one is the most suspicious one S(x_i,wt) =x_i^|wt

—verify greedily!

• same asapproach 2in motivating one-dimensional data

how to correctsampling biaswith greedy queries?

(10)

Random Then Greedy (RTG)

• random sampling for learningfirst

• greedy for verificationlater

• RTG: one-time switching with parameter

S(x_i,w_t) =

(random(), if t ≤ T x_i^|w_t, otherwise

how to learn faster thanrandom sampling?

(11)

Uncertainty Sampling Then Greedy (USTG)

• active learning: similar tointeractive verificationwith different goals

input

instances active

learner

human experts

output model query

label

• USTG:active learning(byuncertainty sampling) first,greedy for verificationlater

S(x_i,wt) =

( ₁

|x_i^|wt|+1, if t ≤ T x_i^|wt, otherwise

how to do better thanone-time switching?

(12)

Another Related Problem: Contextual Bandit

input context

contextual bandit learner

environment

output total reward action

reward

input

instances interactive verifier

human experts

feedback

interactive verification

= special contextual bandit +verified instances as rewards

(13)

Upper Confidence Bound (UCB)

interactive verification

= special contextual bandit +verified instances as rewards

• contextual bandit: balanceexploration(getting information) and exploitation(getting reward)

• interactive verification: balancelearningandverification

• UCB: borrow idea from a popular contextual bandit algorithm S(x_i,w_t) =x_i^|w_t + α ·confidence on x_i

• α: trade-off parameter betweenexploration(learning) and exploitation(verification)

four approaches to be studied: greedy, RTG, USTG, UCB

(14)

Comparison between the Four

learning intent verification intent switching

greedy none positiveness none

RTG random sampling positiveness one-time USTG active learning positiveness one-time UCB confidence term positiveness dynamic

greedy: special case of RTG ( = 0), USTG ( = 0), UCB (α = 0)

(15)

Data Sets

data set number of instances number of positive instances percentage of positive instances

KDDCup2008 102294 623 0.6%

spambase 4601 1813 39.4%

a1a 1605 395 24.6%

cod-rna 59535 19845 33.3%

mushrooms 8124 3916 48.2%

w2a 3470 107 3%

—resampled with 1000 negative andP positive instances

will show

1 P

T

X

t=1

[Y (qt) =1]

under T = 100

(16)

Effect of

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55

ε

performance

USTG RTG

(a) KDDCup2008 with P = 100

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6

ε

performance

USTG RTG

(b) KDDCup2008 with P = 50

‘naive’greedy ( = 0) better thanRTGandUSTG, why?

(17)

Good Properties of Greedy

Case 1: positive instance selected

• successfulverification :-)

Case 2: negative instance selected

• ‘most unexpected’ negative instance

• usually helplearninga lot:-)

greedy approach happy ‘:-)’ either way

(18)

Artificial Data that Fails Greedy

−10 −8 −6 −4 −2 0 2 4 6 8 10

−10

−5 0 5 10

• twopositiveclusters and one bignegativecluster

• greedyignoresbottom cluster:negative instances selected doesn’t help learning

need to query ‘far-away’ (less confident) instances

—UCB to the rescue

(19)

Comparison between UCB and Greedy

P = 50 KDDCup2008 spambase a1a

UCB (α = 0.2) 0.5968 ± 0.0031 0.7306 ± 0.0020 0.3915 ± 0.0034 greedy 0.5868 ± 0.0040 0.7467 ± 0.0024 0.3883 ± 0.0034

comparison × 4

P = 50 cod-rna mushrooms w2a

UCB (α = 0.2) 0.7333 ± 0.0024 0.9776 ± 0.0007 0.6160 ± 0.0024 greedy 0.7249 ± 0.0027 0.9710 ± 0.0014 0.5944 ± 0.0030

comparison

UCBwins( ) often, across data sets and P

(20)

Conclusion

• formulated a novel problemof interactive verification

• connectedthe problem to active learning andcontextual bandit

• studieda simple solutiongreedy

• proposeda promising solutionUCBvia contextual bandit

• validatedthat greedy and UCB lead topromising performance

(21)

Self-Advertisement: TAAI 2014 in Taipei

2014 Conference on Technologies and Applications of Artificial Intelligence

TAAI 2014 Cal l f or Paper

Introduction

* Agents

* AIApplications

* AIArchitectures

* Automated Reasoning

* Cloud Computing in AI

* Cognitive Modeling

* ComputerGames

* ComputerSupported Collaborative Learning & Personalized Learning

* ComputerVision

* Data Mining

* Distributed AI

* Evolutionary Computation & Genetic

* Evolutionary Computation & Genetic Algorithms

* Fuzzy Systems

* Hybrid Systems

* Information Retrieval

* IntelligentE-Learning & Tutoring

* IntelligentEnvironments

* IntelligentHuman-ComputerInteraction

* Knowledge-Based Systems

* Knowledge Representation

* Logics in AI

* Machine Learning

* Mobile Intelligence

* NaturalLanguage Processing

* Pattern Recognition

* Planning

* Probabilistic and Uncertain Reasoning

* Problem Solving and Search

* Robotics

* Semantic Web

* SocialComputing

* Speech Recognition and Synthesis

* Ubiquitous IntelligentApplications

* Web Intelligence General Co-Chairs

Program Co-chairs Hsuan-Tien Lin

(NationalTaiwan University) Hsing-Kuo Kenneth Pao Jane Yung-Jen Hsu (NationalTaiwan University) Yuh-Jye Lee

(NationalTaiwan University ofScience and Technology)

Key Dates

Submission welcomed! Thank you

Chou & Lin (NTU CSIE) Machine Learning Approaches for Interactive Verification 20/20

Machine Learning Approaches for Interactive Veriﬁcation

Machine Learning Approaches for Interactive Verification

Breast Cancer Screening

What If Too Many Positive Predictions?

Learner versus Verifier

Motivation: One-Dimensional Separable Data

Interactive Verification Problem

Our Contribution

Simple Framework for Interactive Verification

Greedy

Random Then Greedy (RTG)

Uncertainty Sampling Then Greedy (USTG)

Another Related Problem: Contextual Bandit

Upper Confidence Bound (UCB)

Comparison between the Four

Data Sets

Effect of 

Good Properties of Greedy

Artificial Data that Fails Greedy

Comparison between UCB and Greedy

Conclusion

Self-Advertisement: TAAI 2014 in Taipei

TAAI 2014 Cal l f or Paper

Effect of