國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
app easily.
4.3 Apps in Weather Category
One of the app in "Weather" category analyzed by AppCAT is Clear Day - Weather HD which provides pretty videos depicting weather conditions, and a quick view for multiple cities weather. The review analysis result is shown in table 7 AppCAT finds out that this app has some issue of advertisement. And the user interface of this app, is good for users (about 0.44 points). but with some complains (-0.06 points). We can also figure out from the bar chart 6. The app have great design and good performance but also with advertisement issue. The advertisement issue is complained by users. It is still a highly-recommended app for user to download.
Figure 6: Clear Day - Weather HD comment analysis result
Another app in weather category is Weatherwise, it is an app with creative animated weather scenes, photographers. The following review:
Cute and simple and no ads
It implies that the app has no advertisements. AppCAT also identifies that it has little performance issue. So it is a good app to download.
‧
Table 7: AppCAT result of Clear Day - Weather HD - Live Weather Forecast with NOAA Radar Free(track id 412489722)
Feature name Original rating Sentiment Sentence
PERFORMANCE 3 0.54 Good performance, good data, and
appealing presentation... at first.
PERFORMANCE 2 0.25 Also, ever since the update, the
program crashes upon first opening.
PRICE 2 0.55 I’m happy to buy stuff but this doesn’t
seem to have functionality worth paying for.
PRICE 5 0.22
It is good because it tells you how the weather is going to be but bad because you haft to buy to get more weather news at the starting you only can have 1 news and that is new your and then you can look for 1 more than you have to buy the rest well it is good for...
NEW YOURK
PRICE 3 0.42
I love this app, and want to buy the full version, but it crashes on
some of the maps - it always crashes when I try to play the animation on the precipitation map.
UI/UX 5 0.81 All the best in easy to access information
packaged with great design!
UI/UX 5 -0.12 You can change your settings & add
other locations to see what the temperature is.
UI/UX 5 0.18 Clean and simple design.
ADVERTISEMENT 1 0.00 I didn’t like the ads either.
ADVERTISEMENT 4 0.12 I recommend it...wish there were
less ads and pop-ups, but it’s free.
ADVERTISEMENT 3 0.20 No radar, no multiple cities in free version,you do get two plus a lot of ads.
‧
Table 8: AppCAT result of Weatherwise(track id 420954273) Feature name Original rating Sentiment Sentence
PERFORMANCE 4 0.40 Good detail and more accurate than
local network forecasts.
PERFORMANCE 5 0.35 Continue the good work as always..Weatherwise crash when I open my iPhone please
fix it ..
PERFORMANCE 2 0.20 glad that the icon has changed to
the old one .... but the app crash when i open it !!
PRICE 5 0.30 I might buy a theme pack or something
it’s pretty cool
PRICE 2 -0.12 You will have to buy the themes again
from the other device.
PRICE 2 -0.12
Not like the other apps where if you buy any additional stuff from the app you can apply everything to the other device.
UI/UX 5 0.80 Great design
UI/UX 5 0.33
Simply stated, this is a stunning app that combines thoughtful design with accurate weather and a variety of themes (the default is still
my favourite) which reflect the changing weather conditions and time of day.
UI/UX 5 0.62 Love the artwork and interface
design!
ADVERTISEMENT 5 0.25 Cute and simple and no ads!!
ADVERTISEMENT 5 0.00 There are no ads in the basic version.
ADVERTISEMENT 5 0.00 Nothing drives me more angry than
ads when all I want is the temp today.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 7: Weatherwise comment analysis result
5 Validation with AppReco
In this section we compare the result of AppCAT review analysis to AppReco to validate the experiment process. AppReco is a system that through static analysis to find out the hidden system call and analyze the behavior of mobile applications. We can compare the result of AppReco and AppCAT with respect to same app.
Take an analysis result in AppReco as an example: "Clear Day - Weather HD", it has 2.0 score in privacy. While AppCAT didn’t report any privacy issue, it means that this app has some hidden privacy concerns that user do not notice. And the advertisement score output by AppReco is 0.67. It is reported by AppCAT that Advertisement score is also low due to the serious advertisement issue.
It is interesting that most app in AppCAT do not have a privacy score. AppReco can be a good approach to identify the hidden threat in apps while user doesn’t know what api the app is calling.
The second case is comparison of two photo application, "Pic Stitch" and "Pic Col-lage". Result displayed in table 9 and table 10.
As table shown that the higher the privacy risk in AppReco, the more danger the
‧
app is. In this aspect, "Pic Collage" has a 0.75 in privacy. While user give this app a lower rate on privacy in AppCAT, it is not surprising that two result points to the same conclusion.
Table 9: Comparison of "Pic Stitch" and "Pic Collage" in AppReco App Name Privacy Third Party SDK Advertisement Total Risk
Pic Stitch 0.26 0.7 0 0.96
Pic Collage 0.75 0.7 0 1.45
Table 10: Comparison of "Pic Stitch" and "Pic Collage" in AppCAT App Name Performance Price UI/UX Advertisement Privacy
Pic Stitch 0.43 0.54 0 0.17 0.43
Pic Collage 0.42 0.44 0.55 -1.0 0
6 Conclusion
We present AppCAT, a systematic way of analyzing mobile application reviews. It can automatically analyze reviews in App Store and based on the feature defined and extended by word2vec then classify reviews and perform sentiment analysis on each comment sen-tences. As a result we can plot radar charts to each application for user reference when downloading.
There’reome sentences that contains negation like "This app doesn’t care about user experience." Such sentence sometimes can’t be recognized by our methodology. Unlike blogs and movie review posts, reviews to mobile applications are usually brief in content.
Thus, it contains less information. And sometimes the reviews don’t have a comment subject. For future work, the sentiment analysis part can be tuned by other methods like deep learning to improve the accuracy of sentiment polarity.
References
[1] app-attention-span-research-report-1.pdf. https://www.appdynamics.com/media/
uploaded-files/1425406960/app-attention-span-research-report-1.pdf.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(Accessed on 03/03/2016).
[2] AppBrainStat. http://www.appbrain.com/stats. Accessed: 2015-06-12.
[3] Gartner Says Worldwide Sales of Cellular-Embedded Mobile PCs, Tablets and Mo-bile Hot Spot Devices Will Exceed 112 Million in 2015. http://www.gartner.com/
newsroom/id/3064718. Accessed: 2015-06-12.
[4] Github - vacuumv/appcat: Mobile application in ios review semtiment analysis.
https://github.com/vacuumv/AppCat. (Visited on 01/27/2016).
[5] Google play android. https://play.google.com/
store/apps?utm_source=apac_med&utm_medium=hasem&utm_
content=May2516&utm_campaign=Evergreen&pcampaignid=
MKT-DR-apac-tw-all-med-hasem-py-Evergreen-May2516-1-BKWS%7c%2EHASEM_
kwid_43700011378110954&gclid=Cj0KEQjwhZm7BRCUyfS6ho2VjOEBEiQAumpGMt367Hff9cpZOExC345iypPiYUsoLkeNX88vvM9OdVgaAh7E8P8HAQ&
gclsrc=aw.ds#/now. (Accessed on 06/20/2016).
[6] https://itunes.apple.com/rss/customerreviews/id=340233007/sortby=mostrecent/page=1/json.
https://itunes.apple.com/rss/customerreviews/id=340233007/sortby=
mostrecent/page=1/json. (Accessed on 06/20/2016).
[7] Improving hls on android - jw player sdk for android 1.2. https://www.jwplayer.
com/blog/improving-hls-android-sdk-1-2/. (Accessed on 03/03/2016).
[8] itunes apple. http://www.apple.com/tw/itunes/charts/free-apps/. (Accessed on 06/20/2016).
[9] Product/service features and benefits - entrepreneur-ship.org. http://www.entrepreneurship.org/resource-center/
productservice-features-and-benefits.aspx. (Accessed on 06/20/2016).
[10] The Statistics Portal. http://www.statista.com/statistics/276623/
number-of-apps-available-in-leading-app-stores. Accessed: 2015-06-12.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
[11] What app users care about when sharing personal data: Permissions – martindale.com. http://www.martindale.com/intellectual-property-law/
article_Mintz-Levin-Cohn-Ferris-Glovsky-Popeo-PC_2220436.htm. (Accessed on 03/03/2016).
[12] Lillian Lee Bo Pang. Opinion Mining and Sentiment Analysis. Foundations and TrendsR in Information Retrieval, 2(1):1–135, 2008.
[13] Xing Fang and Justin Zhan. Sentiment analysis using product review data. Journal of Big Data, 2(1):5, 2015.
[14] Leonard Hoon, Rajesh Vasa, Jean-guy Schneider, and John Grundy. An Analysis of the Mobile App Review Landscape : Trends and Implications. pages 1–23, 2013.
[15] Leonard Hoon, Rajesh Vasa, Jean-Guy Schneider, and John Grundy. An Analysis of the Mobile App Review Landscape: Trends and Implications. Technical report, Department of Computer Science and Software Engineering, Swinburne University of Technology, July 2013.
[16] Su Su Htay and Khin Thidar Lynn. Extracting product features and opinion words using pattern knowledge in customer reviews. TheScientificWorldJournal, 2013:394758, 2013.
[17] Khairullah Khan, Baharum Baharudin, and Aurangzeb Khan. Identifying product features from customer reviews using hybrid patterns. International Arab Journal of Information Technology, 11(3):281–286, 2014.
[18] Efthymios Kouloumpis, Theresa Wilson, and Johanna Moore. Twitter Sentiment Analysis : The Good the Bad and the OMG ! pages 538–541, 2011.
[19] Bing Liu. Sentiment Analysis and Opinion Mining. Synthesis Lectures on Human Language Technologies, 5(1):1–167, 2012.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
[20] George A. Miller. Wordnet: A lexical database for english. COMMUNICATIONS OF THE ACM, 38:39–41, 1995.
[21] Subhabrata Mukherjee and Pushpak Bhattacharyya. Feature specific sentiment anal-ysis for product reviews. Computational Linguistics and Intelligent Text Processing, pages 1–12, 2012.
[22] Rebecca Passonneau. Sentiment Analysis of Twitter Data.
[23] Ana Maria Popescu and Orena Etzioni. Extracting product features and opinions from reviews. Natural Language Processing and Text Mining, (October):9–28, 2007.
[24] Ana-Maria Popescu and Orena Etzioni. Extracting Product Features and Opinions from Reviews, pages 9–28. Springer London, London, 2007.
[25] Phong Minh Vu, Tam The Nguyen, Hung Viet Pham, and Tung Thanh Nguyen.
Mining User Opinions in Mobile App Reviews : A Keyword-based Approach. 2015.