國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Chapter 6 CONCLUSION
This work presented a posting recommender, providing relative and useful ideas and features, to supports users constructing their questions during an asking process. Users can decide to accept or ignore outputted recommendations. We expect our innovative design is able to make UGCs informative and clear to attract experts to give back feasible answers on a healthcare Q&A online forum. Effectiveness, efficiency, and circumstance are standards we adopted to evaluate the usability of a posting recommender (RQ1). Combing the result with RQ1, we evaluate the feasibility of processed post to see if experts rate higher points (RQ2). We summarize the important results for the two research questions as the followings:
RQ1: From H1 measurement’s perspective, participants produce more medical-related features
and have lower possibility to add descriptions into a post under the Word2Vec recommender.Only medical-related features fit effectiveness standard in a positive way. Second, according to the efficiency standard, participants produce fewer medical-related features with Word2Vec than with WordNet. The result does fit its hypothesis but is different from what we expected.
Thirdly, participants used more time on word embedding with a daily task than baseline with a daily task, word embedding with a class task than baseline with a class task, and semantic
with a daily task than baseline with a daily task. Using recommender takes more time obviously.
RQ2: From the labeled “True” description perspective, participants using RSs including word
embedding and semantic can get better complete or clear points.In RQ1 work, although most of the results don’t meet our expectation (e.g. the performance of word embedding will be better than semantic), most hypotheses still established. Therefore, we conclude that the posting recommender does have significant effects on the asking process when formulating questions. Though, whether the new design helps a lot on healthcare Q&A online forums still needs more investigations as evidences. In RQ2 work, simply the significant effect on having a description to get higher points is not enough for stating the usability of a posting recommender. So, after executing the interaction evaluation, we found if a post with more information (having description) under a posting recommender, experts may be attracted because it is easy to understand askers’ situation and then support them as soon as possible.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
This research reveals current Q&A forum recommendation systems have already reached a limit because, for a long time, they only present relative questions based on the same words and send query requests to who may help askers. These supportive methods aren’t so feasible when posts are difficult for the system to classify and users think not to bother people who are reluctant to answer. Not to mention, most Q&A online forums do nothing about post actions in the asking process. In addition, the existence of unanswered posts underlines the necessity to optimize the posting process. After the user study, we found participating more in the asking process via a posting recommender is possible to change the old pattern of posting. Askers are also willing to be supported by the feature RS when formulating questions in an unfamiliar domain. Whether recommended features are able to be adopted directly or relative enough to modify posts conceptually, our RS does try to give concrete and possible ideas to askers. So, if the posting RS performs smoothly, this thesis creates a new manipulation in the posting area.
Next, our recommender can support non-native English speakers to write purposed English sentences clearly. English learners are usually not familiar with the language culture and wordings used in everyday conversation, so it is hard for them to propose qualified posts all the time. Thus, they may prefer to ask questions on domestic and mother-language platforms.
However, we now live in a globalized society that can absorb knowledge all over the world anytime. It is unreasonable that simply finding solutions from a person using the same language as us. We hope the posting recommender supports non-natives formulate posts well and get solutions faster than before, stuck by unofficial or strange English wordings.
The third contribution of our recommendation mechanism is to enlarge the applied domains, not limited in the healthcare area. Take e-commerce for example, when people are purchasing products that they are unfamiliar, it is common for them to ask details before and after buying.
If there is a system formulate questions better, processed posts can be more correctly arranged to match FAQs. If FAQs are still not able to solve their problems, websites then present existent posts from other askers. The advanced posting recommender can try to resolve questions before posting to the Q&A online forum. So, the unanswered rate goes down and the possibility of getting solutions goes up. Any industry who needs to deal with queries frequently are suitable for much more participating in the users’ asking process.
In the future, if it is necessary to perform the user study again, we think the number of participants should increase, the illness selection should be reconsidered, data resources to
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
make a recommender should increase, and the design of recommendation presentation is better to be more user-friendly. Without a doubt, more participants join our study brings more reliable results. Second, some participants commented the picked illness is so general that they don’t have to execute a RS to complete a post. This feedback is totally different from our pilot study, so we should rethink the selection. Among all four adjustments, collecting more data from healthcare forums is the most direct way to improve the performance of posting recommenders.
But what kind of data resources should be selected to build the posting recommender? If input resources (exist posts on online forums) are low quality, which means fewer users have replied, the quality of recommender will be low or high? Therefore, deciding to train models from high quality posts is one of choices to enhance our recommender’s usefulness.
Another process we should remember to add next time is the quality assessment of our posting recommender. The brief evaluation now may be one of the reasons that the final result of word embedding is poorer than semantic or baseline (e.g. only executing simulated examples to see if the RS is feasible to provide ideas). Therefore, whether the data resource of recommenders is revised to be higher quality or still contains total exist posts, we plan to employ a method to determine the quality of posting recommenders. One hundred posts were fetched randomly from the data resource of the posting recommender (e.g. questions from the WebMD website).
Features from the first sentence of every post will be extracted to put into our Word2Vec model.
Then we can observe if produced relative features have any matches with features presenting in the following sentences of every post. This outcome is defined as accuracy. We will calculate an average of accuracy from one-hundred sets to judge the quality of a word embedding RS.
However, WordNet model is implemented by the Python package, so it doesn’t have data resources to analyze like Word2Vec. But we can still calculate accuracy by the same method using simulated posts in the user study to evaluate if this semantic RS is suitable for participants.
Besides, further study with eye-tracking augmentation may be useful to learn more about the interaction between orders of decision-making and types of posting recommenders. We hope to find more commonalities among the statistical result from users’ behavior in RSs, interaction records from eye-tracking tool, and feedbacks from questionnaires and interviews.