CHPATER 6 EVALUATION
6.1 P ROPOSITIONS
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
93
In the precious chapters, we defined the environment from the SME’s point of view, and then combined the foundation theories and methodologies to design and build the system artifact to help the SMEs to get the turnaround from the current predicament. Mentioned in the IS research, the utility, quality, and efficacy of a design artifact must be manifested rigorously via well-completed evaluation methods.
Evaluation is a pivotal constituent of the entire research process (Hevner et al., 2004).
Following the guidelines of the IS research, we need to justify and evaluate our service system in the next step.
The following are the four sections: In the first section we depict some propositions which respond to the research questions proposed in the Chapter 1 and will be verified in this research. Next, the assumptions and details of experimental data and the related propositions will be provided in the second section. The third part capitalizes on the result of the experiments to optimize the parameters of the system mechanism and answer the research questions presented in the Chapter 1. In the final section, we will provide the summary of the experimental results and conclude with some discussion.
6.1 Propositions
This section we will be based on the IS research framework to design the evaluation methods. The scholar summarized five evaluation methods which use methodologies available in the knowledge base, and proposed that the postgraduate who need to evaluate the new artifact must select the appropriate evaluation methods which match with the designed artifact and the selected evaluation metrics (Hevner et al., 2004). The summarized five evaluation methods as fellow: Observational, Analytical, Experimental, Testing and Descriptive. Owing to two research questions, the accuracy of the recommended advertising mini story and the effectiveness of the
‧
methods to take on the examination. First, based on the general recommender system we can use the traditional methods to test the accuracy of the recommender system, followed by observing the human-computer interactions to examine the effectiveness of the advertising. The above-mentioned directions include two evaluation methods concluded by Hevner (2004): Observational and Analytical. Based on the observation of field study we take some statistics to do analysis and support our thesis argument.The reason why we eliminate the other three methods is that it’s not an important issue in this thesis about the Testing (the point here is the validity of the system, not the data flow (White Box) and the web security (Black Box)), Experimental (the major test object of the recommender system is people, so we can’t simulate the reaction from people inspired by the advertising) and Descriptive (the statistics analysis and field study are more convincing than the scenarios way).
As noted in previous chapter, we want to construct a mechanism which can recommend appropriate advertising mini story to the opportune user, and stimulate user to coordinate the entire ImageCons system’s procedure to help them to do service innovation pass through this step. Hence, we can simply put our advertising mini story automatic generator system into the recommender system. And then, according to the method which is taken to examine the effectiveness of the recommender system, we take the same way to test and verify the advertising mini story generation recommender system to ensure the story can be designate to the suitable person to achieve the incentive function.
Recommender systems are intelligent applications designed to identify the interesting products or services of each user and support them to make decision in the era of information explosion (Ricci & Werthner, 2006). There are two phases of constructing recommender system: user-model construction and recommendation
‧
generation (Ricci & Werthner, 2006). The user-model construction part utilizes the collection of previous user-system interactions to construct the structured description of user’s demand and preference, and then the recommendation generation part based on this structure model to make the recommendation (Ricci & Werthner, 2006).
Similarly, the original intention of advertising mini story generator system is to recommend distinctive story frame to each suitable user and customer-made in the detail of story for each user as mentioned before. Therefore we may be able to classify our system as one kind of recommender systems. Following the two phases of constructing recommender system, first we need to build a user-model structure to describe the need and preference of different types of user. Instead of using the collection data from the database as the traditional recommender system way, we use the Maslow's Hierarchy of Needs to model users. Because the “service” which we want to recommend is the incentives advertising story, we couldn’t only base on the human-computer interaction to judge which story is suitable for them. In other words, we couldn’t only observe what they purchased in the past (as did by many recommender systems) to determine their hierarchy of needs to make the most appropriate excitation. On the other hand, we base on the Maslow's Hierarchy of Needs to create the advertising mini story as we mentioned before. Therefore, we decide to replace the traditional way with the Maslow's Hierarchy of Needs way to construct the first phases of constructing recommender system. Thus, we have the first proposition formed.
Proposition 1: The SMEs Classification Module of advertising mini story generator system which was developed based on the Maslow's Hierarchy of Needs could be used to construct the user-model structure as the recommender systems.
Reviewing the mechanism of the SMEs Classification Module of advertising
‧
operating conditions, the opposite position with other competitors and the satisfaction of the demand for the innovation. Based on this information, we give each SME one score to classify. In other word, the SMEs Classification Module establishes its function based on the user’s behavior. In this idea, we need to test and verify the level of this score which we give to SMEs to make classify whether the level of score conform the Maslow's Hierarchy of Needs. We then have the following propositions. Proposition 1-A: It is reasonable and feasible to establish the user-model which is based on the Maslow's Hierarchy of Needs to model the user’s behavior and information.
After confirming the advertising mini story generator system is a recommender system in a certain extent, we can inspect the effectiveness of the system’s recommended from the recommender system’s point of view. Because of the position of advertising mini story generator system in the ImageCons, the most important thing is to examine the SMEs’ moving degree to make sure our user will coordinate with
The proposition of above-mentioned is from the system’s point of view to inspect the effectiveness of the advertising mini story generator system. On the other, based on the theory we mentioned in the previous chapter, the conjunction between the similarity of the story and the degree of incentive. The interrelated plight is an
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
97
important element when someone wants to persuade other (Rogers, 1962). The proposition for this issue is put in the following.
Proposition 3: The SMEs will have more incentives when there are more their own related elements in the story.
In the next section, it outlines the assumptions made for revealing the limitations of the mechanism and the presupposition based on some theory as well as the experimental data set to test the propositions depicted in the previous section.