Matrix Factorization with Domain Knowledge and Behavioral Patterns for Intent Modeling
Yun-Nung (Vivian) Chen, Ming Sun, and Alexander I. Rudnicky
The Task
• In a smart-phone intelligent assistant setting (e.g. requesting an app), the feature-enriched MF-SLU can handle users’ open domain intents by
returning relevant apps that provide desired functionality either locally
available or by suggesting installation of suitable apps and doing so in an
unsupervised way.
• The framework can extend to
incorporate personal behavior history for improving a system’s ability to
assist users pursuing personalized multi-app activities.
• The effectiveness of the feature- enriched MF-SLU model can be
shown in different domains, indicating good generality and providing a
reasonable direction for the future work.
Motivations
o An typical SDS has two main challenges:
1) Predefined ontology: the domain ontology is required to support the corresponding functions
2) Language ambiguity: same utterance may infer different intents during different situations
o Structured knowledge resources are available (e.g. Freebase, Wikipedia, FrameNet) and may provide semantic information
o Users’ behavioral patterns may help disambiguate the current intents o Hidden semantics help infer the relation between different features
Feature-Enriched MF-SLU: Spoken Language Understanding by Matrix Factorization
• Data: speech data collected from the users with the intents from 13
frequently accessed domains in Google Play (WER = 19.8%)
MF learns a set of well-ranked intents per utterance.
Lexical Matrix
• Main idea: utilize manual written app description because it
should describe the app’s functionality
• Main idea: retrieve the apps
that are more likely to support users’ requests for self-training
The feature-enriched MF-SLU can benefit from both hidden information modeled by MF and enriched semantics including structured knowledge and behavioral patterns to improve Intent prediction.
Experiments
Conclusions
1
Lexical Intent (App)
photo tell check camera IM
take this photo
tell vivian this is me in the lab
CAMERA Train IM
Dialogue
check my grades on website send an email to professor
…
CHROME EMAIL
send null camera
.85 CAMERA
IM
…
1
1
1 1
1
1 .70
chrome
1
1 1
1
1 1
chrome email 1
1
1
1
.95
.80 .55
User Utterance Intended
App
Reasoning with MF
Test Dialogue
take a photo of this send it to alice
…
Behavioral
1
Enriched Semantics communication
.90 1
1
Utterance 1 i would like to contact alex
Lexical Intent (App)
… …
contact email message Gmail Outlook Skype
Test
.90
Reasoning with MF
Train
… your email, calendar, contacts…
… check and send emails, msgs … Outlook
Gmail
IR for app candidates
App Desc
Self-Train Utterance
Test
Utterance
1 1
1
1
1
1
1
1 1
1
1 .90 .85 .97 .95
Semantics Enrichment Utterance 1 i would like to contact alex
…
1
1
Enriched Semantics Matrix Intent Matrix
• Main idea: slot types and word embeddings help imply semantics for expanding domain knowledge
Feature Matrix ASR Transcripts
LM / MLR MF-SLU LM / MLR MF-SLU
Single-Turn Word Observation 25.1 29.2 (+16.2%) 26.1 30.4 (+16.4%) + Type-Enriched Semantics 31.5 32.2 (+2.1%) 32.9 34.0 (+3.4%) Multi-Turn Word Observation 52.1 52.7 (+1.2%) 55.5 55.4 (-0.2%)
+ Behavioral Patterns 53.9 55.7 (+3.3%) 56.6 57.7 (+1.9%)
Approaches: Feature-Enriched MF-SLU
o Enrich semantics with the structured knowledge or behavioral patterns for improving intent prediction
o Unify the human written knowledge and automatically inferred information in a matrix and predict user intents in the mean time
Results
o Feature-enriched MF-SLU benefits from hidden information and rich features, and then outperforms the baselines for both single-turn
requests and multi-turn interactions.
send to vivian
v.s.
Email? Message?
Communication
previous turn
Challenge: language ambiguity 1) User preference
2) App-level contexts
• Modeling Implicit Feedback:
1
𝑓
+𝑓
−𝑓
−𝑢
𝑥
Matrix Factorization
• Objective:
Experiment 1: Single-Turn Request
Experiment 2: Multi-Turn Interaction
• Entity Type from Structured Knowledge (e.g. Wikipedia/Freebase)
Q: play lady gaga’s bad romance
… is an American singer, songwriter, and actress. … is a song by American singer …
Chen et al., "Leveraging Behavioral Patterns of Mobile Applications for Personalized Spoken Language Understanding," in Proc. of ICMI, 2015.
Data Available at http://AppDialogue.com/
Chen and Rudnicky, "Dynamically Supporting Unexplored Domains in Conversational Interactions by Enriching Semantics with Neural Word Embeddings," in Proc. of SLT, 2014.
