U NSUPERVISED U SER I NTENT M ODELING BY F EATURE -E NRICHED M ATRIX F ACTORIZATION
Yun-Nung (Vivian) Chen, Ming Sun, Alexander I. Rudnicky, and Anatole Gershman Summary
Challenge of typical SDS: Predefined Ontology & Hidden Semantics
1) Predefined domain ontology is required to support corresponding functionality
Structured knowledge resources are available (e.g. Freebase, Wikipedia, FrameNet) and may provide semantic information
2) Hidden semantics may contain important semantics
Implicit information helps infer feature relations
Approach: Feature-Enriched MF-SLU
o Enrich semantics with the structured knowledge for improving intent prediction o A single matrix integrating different-level knowledge for reasoning and
prediction simultaneously
Result
o Feature-enriched MF-SLU benefits from hidden information and rich features, and outperforms the baseline that uses a language-modeling retrieval model.
1. Feature-Enriched MF-SLU: Spoken Language Understanding by Matrix Factorization
Reasoning via MF for SLU
Enriched semantics significantly improve the performance for intent modeling
Conclusion 3. Experiments
Dataset: single-turn request with intents below
Evaluation Metrics
o Mean Average Precision (MAP) o Precision at K (P@K)
• MAP for Intent Modeling
• P@10 for Intent Modeling
• Data: speech data collected from users, with intents from 13 frequently accessed domains in Google Play (WER = 19.8%)
Lexical Matrix
• Main idea: use manually
authored app description as it should describe the app’s
functionality
• Main idea: retrieve the apps
that are most likely to support users’ requests, for self-training
1
Enriched Semantics communication
.90 1
1
Utterance 1 i would like to contact alex
Lexical Intent (App)
… …
contact email message Gmail Outlook Skype
Test
.90
Reasoning with MF
Train
… your email, calendar, contacts…
… check and send emails, msgs … Outlook
Gmail
IR for app candidates
App Desc
Self-Train Utterance
Test
Utterance
1 1
1
1
1
1
1
1 1
1
1 .90 .85 .97 .95
Semantics Enrichment
Utterance 1 i would like to contact alex
…
1
1
Enriched Semantics Matrix Intent Matrix
• Main idea: slot types and word embeddings help infer semantics for expanding domain knowledge
• Entity Type from Structured Knowledge (e.g. Wikipedia/Freebase)
Q : play lady gaga’s bad romance
… is an American singer, songwriter, and actress. … is a song by American singer …
Chen and Rudnicky, "Dynamically Supporting Unexplored Domains in Conversational Interactions by Enriching Semantics with Neural Word Embeddings," in Proc. of SLT, 2014.
MF learns a set of well-ranked intents per utterance.
• Modeling Implicit Feedback:
1
𝑓+ 𝑓− 𝑓−
𝑢
𝑥
2. Model Learning by Matrix Factorization
• Objective:
Lady Gaga Bad Romance
Trailer of Iron Man 3
Alex
Alex
Alex
“I can Alex
meet”
…
“I graduated”
…
…
? CMU
?
CMU
English: university
Chinese: ?
1. music listening
2. video watching 7. post to social websites
4. video chat
5. send an email
8. share the photo 6. text
9. share the video 3. make a phone call
10. navigation
11. address request
12. translation
13. read the book
Feature Matrix (MAP) ASR Transcripts
LM MF-SLU LM MF-SLU
Word Observation 25.1 29.2 (+16.2%) 26.1 30.4 (+16.4%) + Embedding-Enriched Semantics 32.0 34.2 (+6.8%) 33.3 33.3 (-0.2%) + Type-Embedding-Enriched Semantics 31.5 32.2 (+2.1%) 32.9 34.0 (+3.4%)
Feature Matrix (P@10) ASR Transcripts
LM MF-SLU LM MF-SLU
Word Observation 28.6 29.5 (+3.4%) 29.2 30.1 (+2.8%)
+ Embedding-Enriched Semantics 31.2 32.5 (+4.3%) 32.0 33.0 (+3.4%) + Type-Embedding-Enriched Semantics 31.3 30.6 (-2.3%) 32.5 34.7 (+6.8%)
Type information inferred from ASR results may not be accurate enough; noisy enriched information could be degrading performance.
When there are no recognition errors, accurate type information benefits performance.
model parameters
• We propose an MF approach to learn user intents based on rich feature patterns from
multiple modalities, including app descriptions, automatically acquired knowledge and user
utterances.
• In a smart-phone intelligent assistant setting (e.g. requesting an app), the feature-enriched MF-SLU can handle users’ open domain
intents by returning relevant apps that provide desired functionality either locally available or by suggesting installation of suitable apps in an unsupervised way.
• The framework can flexibly extend to incorporate different-level features for
improving a system’s ability to assist users pursuing personalized multi-app activities.
• The effectiveness of the feature-enriched MF-
SLU model can be shown for different domains, indicating good generality and provides a
promising direction for future work.
please contact vivian Skype, Hangout, etc.
Communication
The feature-enriched MF-SLU can benefit from both
1) hidden information modeled by MF
2) enriched semantics including structured knowledge from
different modalities to improve Intent
prediction.
