Unsupervised Induction and Filling of Semantic Slot for Spoken Dialogue Systems Using Frame-Semantic Parsing
Yun-Nung (Vivian) Chen, William Yang Wang, and Alexander I. Rudnicky
2014/02/13 Sphinx Lunch
Best Student Paper Award
@ 2013 IEEE Workshop on Automatic Speech Recognition and Understanding – Dec. 9-12, 2013
Overview
Introduction Approach Experiments Conclusion
Overview
Introduction Approach Experiments Conclusion
In 2011…
“… I have long been looking for good
knowledgeable student to take on the task of getting conversational agents to move beyond prerecorded interaction (or clunky spoken dialog systems) …”
“… Wow! Sounds like they know how to achieve the ultimate machine
intelligence at CMU…”
Spoken Language Understanding (SLU)
SLU in dialogue systems
SLU maps NL inputs to semantic forms
“I would like to go to Shadyside Tuesday.”
Semantic frames, slots, & values
often manually defined by domain experts or developers.
location: Shadyside date: Tuesday
What are the problems?
Problems with Predefined Slots
Generalization: may not generalize to real-world users.
Bias propagation: can bias subsequent data collection and annotation.
Maintenance: when new data comes in, developers need to start a new round of annotation to analyze the data and update the grammar.
Efficiency: time consuming, and high costs.
Can we automatically induce semantic slots
only using raw audios?
Overview
Introduction Approach Experiments Conclusion
Probabilistic Frame-Semantic Parsing
(Das et al., 2010; 2013) on ASR-transcribed utterances.
First Step
FrameNet
FrameNet (Baker et al., 1998)
a linguistically-principled semantic resource, based on the frame-semantics theory.
Example
Frame (food): contains words referring to items of food.
Frame Element: a descriptor indicates the characteristic of food.
“low fat milk”
“milk” evokes the “food” frame;
Frame-Semantic Parsing
SEMAFOR (Das et al., 2010; 2013)
a state-of-the-art frame-semantics parser,
trained on manually annotated FrameNet
sentences.
can i have a cheap restaurant
Frame: capability FT LU: can FE LU: i
Frame: expensiveness FT LU: cheap
Frame: locale by use FT/FE LU: restaurant
The Panacea?
Unfortunately...
Task: adapting generic frames to task-specific
Good!
Good!
Bad!
As A Ranking Problem
Main idea
Ranking domain-specific concepts higher than generic semantic concepts
can i have a cheap restaurant
Frame: capability FT LU: can FE LU: i
Frame: expensiveness FT LU: cheap
Frame: locale by use FT/FE LU: restaurant
slot candidate
Slot Ranking Model (1/3)
Rank the slot candidates by integrating two scores
the frequency of the slot candidate in the SEMAFOR-parsed corpus
the coherence of slot fillers
slots with higher frequency may be more important domain-specific concepts should focus on fewer topics and be similar to each other
lower coherence in topic space higher coherence in topic space
slot: quantity slot: expensiveness
a one
all three
cheap
expensive inexpensive
Slot Ranking Model (2/3)
Measure coherence by word-level context clustering
For each slot ,
We have corresponding cluster vector
Measure coherence measure by pair-wised cosine similarity
slot candidate: expensiveness corresponding value vectors: “cheap”, “not expensive”
(from the utterances with s
iin the parsing results)
the frequency of words in v
jclustered into cluster k
Slot Ranking Model (3/3)
Spectral clustering
For each word
The approach can be summarized in five steps:
1. Calculate the distance matrix
2. Derive the affinity matrix
3. Generate the graph Laplacian
4. Eigen decomposition of L
5. Perform K-means clustering of eigenvectors
r
i= 1 when w occurs in the i-th utterance r
i= 0 otherwise
Reasons why spectral clustering:
1) can be solved efficiently by standard linear algebra
Assume that the words in the same utterance are related to each other
Overview
Introduction Approach Experiments Conclusion
Experiments
Dataset
Cambridge University SLU corpus (Henderson, 2012)
restaurant recommendation in an in-car setting in Cambridge
WER = 37%
vocabulary size = 1868
2,166 dialogues
15,453 utterances
dialogue slot (total # = 10): addr, area, food, name, phone,
postcode, price range, signature, task, type
Slot Induction Evaluation
MAP of the slot ranking model
measure the quality of induced slots based on induced and reference
slots via the mapping table
Induced Slot Reference Slot
Speak on topic Addr
Part orientational
Area Direction
Locale Part inner outer
Food
Food origin
(NULL) Name
Contacting Phone
Sending Postcode
Commerce scenario
Price range Expensiveness
Range
(NULL) Signature
seeking
Task Desiring
Approach MAP
ASR Manual Frequency 67.31 59.41
K-Means 68.45 59.76 Spectral Clustering 69.36 61.86
The majority of the reference slots used in a real
The mapping table between induced and reference slots
