Result of Controlled experiment 2 with customer

CHAPTER 6 EVALUATION

6.3 C ONTROLLED EXPERIMENT B WITH CUSTOMERS

6.3.3 Result of Controlled experiment 2 with customer

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

Figure 6.3.14 Hours of spending on surfing internet per day

6.3.3 Result of Controlled experiment 2 with customer

 Result for Proposition 2-1:

We collect data of subjects that compare the search targetability SME-original-choose keywords and SME-revised long tail keywords which revise after using long tail keyword service in the engagement site. The customer’s answer scale from 1 to 7 level of agrees (extremely disagree, strongly disagree, disagree, neutral, agree, strongly agree and extremely agree) toward the SME-revised long tail keywords have higher search targetability.

We use the box plot which is a convenient way of graphically depicting groups of numerical data through their quartiles to show our data. The box plot includes five points, including max, min, first, third quartile and average of the data. The top and the bottom are the max and min, the top and the bottom of rectangle are the first and third quarter, and the line in the middle is the average of the data.

above 7 43%

5-7 30%

3-5 21%

1-3 6%

Hours spending on surfing internet per day

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

Figure 6.3.15 SME-original-choose keywords vs SME-revised long tail

keywords search targetability

Data shows that over 75% of people agree with the SME-revised long tail keywords actually have higher search targetability over SME-choosed keywords (See Figure 6.3.15). And also we can see that over 25% of customers extremely agree that the SME-revised long tail keywords actually have higher search targetability over SME-original-choose keywords. Thus, we then see that the SME-revised long tail keywords have better search targetability. To be more rigorous, we also built a hypothesis for proposition 1-1 (See Figure 6.3.16) and run a One-Sample T test to test the hypothesis.

Figure 6.3.16 Hypothesis of the testing for linking service increase in search targetability

1 2 3 4 5 6 7

SME-choosed keyword vs SME-revised long tail keyword targetability

7 Likerts point scale degrees of agree on SMEs revised long tail keywords search targetability is higher than SME- original-choose keywords

Search targetability

𝐻₀: 𝜇 < 4,𝑆𝑀𝐸 −𝑟𝑒𝑣𝑖𝑠𝑒𝑑 𝑘𝑒𝑦𝑤𝑜𝑟𝑑𝑠 𝑠𝑒𝑎𝑟𝑐ℎ 𝑡𝑎𝑟𝑔𝑒𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑙𝑜𝑤𝑒𝑟 𝑡ℎ𝑎𝑛 𝑆𝑀𝐸 −𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙 𝑐ℎ𝑜𝑜𝑠𝑒 𝑘𝑒𝑦𝑤𝑜𝑟𝑑𝑠.

𝐻1:𝜇 ≥ 4,𝑆𝑀𝐸 − 𝑟𝑒𝑣𝑖𝑠𝑒𝑑 𝑘𝑒𝑦𝑤𝑜𝑟𝑑𝑠 𝑠𝑒𝑎𝑟𝑐ℎ 𝑡𝑎𝑟𝑔𝑒𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦 ℎ𝑖𝑔ℎ𝑒𝑟 𝑡ℎ𝑎𝑛 𝑆𝑀𝐸 −𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙 𝑐ℎ𝑜𝑜𝑠𝑒 𝑘𝑒𝑦𝑤𝑜𝑟𝑑𝑠.

μ= SME −revised keywords search targetability over SME−original choose keywords mean

5.97

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

Table 6.3.1 One-Sample T test for Increment of search targetability for keywords One-Sample Test

Test Value = 4

t df Sig. (1-tailed)

10.0 32 0.00005

The result (Table 6.3.1) shows that the t value is 10.0 and its significance level is 0.0001 (p-value = 0.00005), which is below 0.001. Therefore, the null hypothesis H0 is rejected and it means there is statistically significant evidence to support our proposition 1-1. After SMEs revise the keywords, there is significance increase on search targetability.

 Result for Proposition 2-2:

From comparing the image-related and unrelated site for search targetability for the campaign, we collect the data as shown in figure 6.4.17. We originally assume that image-related site will be more search targetability. However, the average score of search targetability preference toward image-related site is only slightly above neutral which shows little difference between the image-related and image-unrelated promotion site for the campaign.

Figure 6.3.17 image-related vs image-unrelated inlink site search targetability

1 2 3 4 5 6 7

image-related vs image-unrelated inlink targetabiltiy 7 point Liker t scale degree of agree on image-related inlink search targetability is higher than image-unrelated inlink

Search targetability

4.33

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

Since image-related and image-unrelated inlink site don’t make much difference in search targetability (we also do the one-sample t-test but failed to support to make difference), we fail to accept this hypothesis.

 Result for Proposition 3:

The proposition we want to prove that the image-related site which we recommended for SMEs to be actually more consistent than image-unrelated with brand and campaign image. Since the consistency is range from 0 – 12, the difference between image-related site consistency and image-unrelated site consistency over campaign and brands will be range from -12 to 12. As we can see the data boxplot in Figure 6.3.18, the difference of image-related link and image-unrelated link is not so much and over 25% of people recognize the image-unrelated link has more consistency over campaign and brands. But still if the consistency difference is bigger than zero, we can conclude that the image-related site is better than image-unrelated site for increasing the brand and partner image consistency.

Figure 6.3.18 Subtraction of image- related inlink over image-unrelated inlink consistency with campaign and brands

-8 -6 -4 -2 0 2 4 6 8 10 12

su b tract io n of rel at ed -i m ag e in li n k o v er im ag e -u n rel at ed im ag e co n si st ency

Subjects select on inlink image with campaign, brands image consistency

1.05

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

To test the concept we run a One-Sample T test to test the hypothesis (See Figure 6.3.19).

Figure 6.3.19 Subtraction of related-image link over image-unrelated consistency

Table 6.3.2 One-Sample T test for Subtraction of related-image link over image-unrelated consistency

One-Sample Test Test Value = 0

t df Sig. (1-tailed)

1.71 32 0.0485

The result (Table 6.3.2) shows that the t value is 1.171 and its significance level is 0.0485 (p-value = 0.00485), which is below 0.05. Therefore, the null hypothesis H0 is rejected and it means there is statistically significant evidence to support our proposition 3 has failed, and we will do further analysis in the discussion part.

 Result for Proposition 4-1:

There are two ways customers can reach the SME’s web pages, one is through search engine, and the other one is through other website linkage. SMEs won’t trigger any engagement for the customers if customers cannot even reach the webpage.

Due to the fact that we don’t have enough collecting data for the engagement level in our experiments, we use the engagement site’s data as an example, long tail

𝐻₀: 𝜇 < 0, 𝑖𝑚𝑎𝑔𝑒− 𝑟𝑒𝑙𝑎𝑡𝑒𝑑 𝑙𝑖𝑛𝑘 𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 𝑤𝑖𝑡ℎ 𝑏𝑟𝑎𝑛𝑑 𝑎𝑛𝑑 𝑐𝑎𝑚𝑝𝑎𝑖𝑔𝑛 𝑖𝑠 𝑙𝑜𝑤𝑒𝑟 𝑡ℎ𝑎𝑛 𝑖𝑚𝑎𝑔𝑒 − 𝑢𝑛𝑟𝑒𝑙𝑎𝑡𝑒𝑑 𝑙𝑖𝑛𝑘𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 𝐻₁:𝜇 ≥ 0, 𝑖𝑚𝑎𝑔𝑒 − 𝑟𝑒𝑙𝑎𝑡𝑒𝑑 𝑙𝑖𝑛𝑘 𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 𝑤𝑖𝑡ℎ 𝑏𝑟𝑎𝑛𝑑 𝑎𝑛𝑑 𝑐𝑎𝑚𝑝𝑎𝑖𝑔𝑛 𝑖𝑠 ℎ𝑖𝑔ℎ𝑒𝑟 𝑡ℎ𝑎𝑛 𝑖𝑚𝑎𝑔𝑒 − 𝑢𝑛𝑟𝑒𝑙𝑎𝑡𝑒𝑑 𝑙𝑖𝑛𝑘 𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 μ= subtraction of 𝑖𝑚𝑎𝑔𝑒 − 𝑟𝑒𝑙𝑎𝑡𝑒𝑑 𝑙𝑖𝑛𝑘 𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 𝑤𝑖𝑡ℎ 𝑏𝑟𝑎𝑛𝑑 𝑎𝑛𝑑 𝑐𝑎𝑚𝑝𝑎𝑖𝑔𝑛 𝑜𝑣𝑒𝑟 𝑖𝑚𝑎𝑔𝑒 − 𝑢𝑛𝑟𝑒𝑙𝑎𝑡𝑒𝑑 𝑙𝑖𝑛𝑘 𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

keyword can help significantly on engagement level. Actually top 10 queries belong to long tail keyword generate over 61% of clicks for search query from total links from google webmaster(See table 6.3.3 more on Appendix B) on our engagement site (http://www.taipeing.net) and they at least have the minimum level of engagement – involvement.

Table 6.3.3 Top 10 queries for engagement site (Taipeing) Queries Clicks

1 環河南路 55

2 台北五金街 11

3 台北銀樓街 10

4 台北燈具店 10

5 台北燈飾街 8

6 台北燈具街 8

7 台北燈飾店 8

8 taipeing 7

9 環河南路五金行 7

10 環河南路五金街 7

 Result for Proposition 4-2:

For keyword search targetability on engagement level:

We compare low search targetability keywords to high search targetability keywords to see if those two have difference on engagement level. In every level, we can see keywords with high search targetability have higher average score on engagement level compare to keywords with low search targetability. The most significant one is the involvement level which we can see that over 25% select extremely agree that they are going to click into the campaign and the average

‧

involvement level score 6.12 on high search targetability is also much higher than the low search targetability’s involvement level 4.87 score. We can then see that search targetability influence on engagement level especially on involvement.

Figure 6.3.20 High search targetability keywords engagement level

Figure 6.3.21 Low search targetability keywords engagement level To prove the proposition for keywords, we develop the hypothesis test (See Figure 6.3.22) and run the two-sample t test to compare:

1 Customer's 7 point Likert scale of aggree to behavior

High search targetability keywords on engagement level

1 Customer's 7 point Likert scale of aggree to behavior

Low search targetability keywords on engagement level

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

Figure 6.3.22 Hypothesis of the testing for average engagement level difference in low and high search targetability keywords

Table 6.3.4 Paired-Sample T test for average engagement level difference in low and high search targetability keywords

Paired-Sample Test t df Sig. (2-tailed) 4.52 32 Pr < 0.0001

The result (Table 6.3.4) shows that the t value is 4.52 and its significance level is Pr < 0.0001 (p-value < 0.0001), which is way below 0.001. Therefore, the null hypothesis H0 is rejected and it means there is statistically significant evidence to support our proposition that high search targetability keywords can generate higher engagement level.

For inlink site search targetability for campaign on engagement level:

We compare low search targetability inlink site for campaign to high search targetability link to see if those two have difference on engagement level (See Figure 6.3.23, 6.3.24). We can tell from the data that, high search targetability inlink site has higher average engagement level compared to low search targetability inlink site.

𝐻₀: 𝜇₁= 𝜇₂ 𝐻1:𝜇1≠ 𝜇2

𝜇1= average enagement level of high search targetability keywords 𝜇₁= average enagement level of low search targetability keywords

‧

Figure 6.3.23 High search targetability inlink site engagement level

Figure 6.3.24 low search targetability inlink site engagement level To prove the proposition for inlink site, we develop the hypothesis test (See Figure 6.3.25) and run the two-sample t test to compare:

Figure 6.3.25 Hypothesis of the testing for average engagement level difference in low and high search targetability inlink site

1 Customer's 7 point Likert scale of aggree to behavior

High search targetability inlink site engagement level

1 Customer's 7 point Likert scale of aggree to behavior

Low search targetability inlink site engagement level

𝐻0: 𝜇1= 𝜇2 𝐻1:𝜇1≠ 𝜇2

𝜇1= average enagement level of high search targetability inlink site 𝜇1= average enagement level of low search targetability inlink site

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

Table 6.3.5 Paired-Sample T test for average engagement level difference in low and high search targetability inlink site

Paired-Sample Test

t df Sig. (2-tailed)

2.34 32 0.0259

The result (Table 6.3.5) shows that the t value is 2.34 and its significance level is 0.0259 (p-value = 0.0259), which is below 0.05. Therefore, the null hypothesis H0 is rejected and it means there is statistically significant evidence to support our proposition that high search targetability inlink site can generate higher engagement level.

 Result for Proposition 4-3:

As mentioned, we want to confirm the inverted-U relationship between image consistency and average engagement level score. Data shows in figure 6.3.26, we can see that the data quite scattered and do not have a particular inverted-U shape relationship between the two parameters. We thus conclude that the proposition not accepted and will be further discuss in discussion and findings.

Figure 6.3.26 Campaign and partner image consistency vs average engagement level

1 2 3 4 5 6 7

0 2 4 6 8 10 12 14

Average Engagement Level

Campaign and partner image consistecy (including inlink site)

Campaign and partner image consistency vs average engagment level

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

 Result for Proposition 5-1:

This proposition is to prove that engagement site average engagement score will positively influence on target site average engagement score. Data shows that the engagement site average engagement score has high positive relationship on target site average engagement score (See Figure 6.3.27).

Figure 6.3.27 Engagement site average engagement score vs target site average engagement score

To further tell how close the relationship between these two parameters, we use Pearson correlation. The Pearson correlation’s value is 0.69151 which is very close to 0.7 (See Figure 6.3.28). If Pearson correlation equal or higher than 0.7, it means that the two parameters has strong positive relationship. Thus we prove that indeed target site engagement level has a positive relationship with engagement site’s engagement level.

Pearson correlation N = 132 Prob > |r| (under H0 ): Rho=0 engagement site engagement level target site engagement level 0.69151

<.0001***

Figure 6.3.28 Pearson correlation between engagement site average engagement score and target site average engagement score

1 2 3 4 5 6 7

target site average engagement score

engagement site average engagement score

Engagement site average engagement score vs target site average engagement score

‧

 Result for Proposition 5-2:

This proposition argues that campaign and brand image consistency will positively influence customer’s engagement behavior for target site. During the experimenting period, we found the contradicting fact with our proposition that most of the customers have almost no different engagement score between different brands. To analyze it, we plot a graph to show score difference between two brands that hold the campaign (See figure 6.3.29). As showing in graph, most of engagement behavior has almost no difference between two brands. Due to the fact that there is almost no difference between brands, thus we conclude that there are no positive relationship between brands and engagement behavior on target site.

Figure 6.3.29 Engagement behavior score difference between brands in one campaign

 Result for Proposition 6-1:

This proposition argues that long tail keyword service increases the customer engagement level in engagement site. To prove this, we compare the engagement level between SME-original-choose keywords and SME-revised keywords (after SMEs using our long tail keyword service in experiment A).We will analyze two kinds of engagement, including engagement level of engagement site and target site (linking from engagement site)

number of subjects for 4 scenarios

Difference of behavior score between brands in one campaign

Engagement behavior score difference between brands in one campaign

involvement interaction intimiacy influence

‧

Proposition 6-1.1 For engagement site engagement level:

As we can see from data (See figure 6.3.30, 6.3.31), these graph are very similart to the Figure 6.3.20 and 6.3.21. It is because 31 out of 33 subjects consider the SME-revised long tail keywords are high targetability keywords. Thus we see the same trend that the SME-revised long tail keywords perform best at the involvement level, which means customers are very likely to click in if they are using the SME-revised keywords as the search query.

Figure 6.3.30 SME-original-choose keywords engagement level on engagement site

Figure 6.3.31 SME-revised long tail keywords engagement level on engagement site Customer's 7 point Likert scale of aggree to behavior

SME-original-choose keyword engagement level on engagement site

Customer's 7 point Likert scale of aggree to behavior

SME-revised long tail keyword engagement level on

engagement site

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

To prove the proposition for revised keywords that using the long tail keyword service on engagement site, we develop the hypothesis test (See Figure 6.3.32) and run the two-sample t test to compare:

Figure 6.3.32 Hypothesis of the testing for average engagement level on engagement sites difference SME-revised and SME-original-choose keywords

Table 6.3.6 Paired-Sample T test for average engagement level difference in SME-revised and SME-original-choose keywords

Paired-Sample Test t df Sig. (2-tailed) 5.14 32 Pr < 0.0001

The result (Table 6.3.6) shows that the t value is 5.14 and its significance level is smaller than 0.0001 (p-value < 0.0001), which is below 0.001. Therefore, the null hypothesis H0 is rejected and it means there is statistically significant evidence to support our proposition that SME-revised keywords can generate higher engagement level on engagement site.

Proposition 6-1.2 For target site (linking from engagement site) engagement level:

As we can see from data for target site (See Figure 6.3.33, 6.3.34), SME-revised long tail keywords makes less difference in involvement but have comparatively higher agreement to behavior in interaction and intimacy level. But all in all, the SME-revised keywords are average higher than SME-original-choose keywords.

𝐻₀: 𝜇₁= 𝜇₂ 𝐻₁:𝜇₁≠ 𝜇₂

𝜇1= average enagement level of SME − revised long tail keywords on engagement site 𝜇₁= average enagement level of SME − original− choosel keywords on engagement site

‧

Figure 6.3.33 SME-original-choose long tail keywords to target site from engagement site on engagement level

Figure 6.3.34 SME-revised long tail keywords to target site from engagement site on engagement level

1 Customer's 7 point Likert scale of aggree to behavior

SME-original-choose long tail keyword to target site from engagement site engagement level

1 Customer's 7 point Likert scale of aggree to behavior

SME-revised long tail keyword to target site from

engagement site engagement level

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

To prove the proposition for revised keywords that using the long tail keyword service on target site, we develop the hypothesis test (See Figure 6.3.35) and run the two-sample t test to compare:

Figure 6.3.35 Hypothesis of the testing for average engagement level on engagement sites difference SME-revised and SME-original-choose keywords

Table 6.3.7 Paired-Sample T test for average engagement level difference in SME-revised and SME-original-choose keywords

Paired-Sample Test t df Sig. (2-tailed) 5.71 32 Pr < 0.0001

The result (Table 6.3.6) shows that the t value is 5.71 and its significance level is smaller than 0.0001 (p-value < 0.0001), which is below 0.001. Therefore, the null hypothesis H0 is rejected and it means there is statistically significant evidence to support our proposition that SME-revised keywords can generate higher engagement level on not only in engagement site but also on target site.

Thus in this proposition 6-1, we conclude that long tail keyword service can increase both engagement level of engagement site or target site (from engagement site).

 Result for Proposition 6-2:

This proposition argues that link building service increases the customer engagement level in engagement site. To prove this, we compare the engagement

𝐻₀: 𝜇₁= 𝜇₂ 𝐻₁:𝜇₁≠ 𝜇₂

𝜇₁= average enagement level of SME − revised long tail keywords on target site 𝜇1= average enagement level of SME − original − choosel keywords on target site

‧

analyze two kinds of engagement for the inlink situations (scenario 3 and 4, See Figure 6.3.8), including engagement level of engagement site and target site (linking from engagement site)

Proposition 6-2.1 For inlink site engagement level:

As we can see from data (See figure 6.3.36, 6.3.37), the average engagement level for image-related link is slightly higher than image-unrelated link, except influence part.We can tell that the image-related link have slightly higher engagement than image-unrelated link

Figure 6.3.36 Image-related link engagement level (for the campaign)

1 Customer's 7 point Likert scale of aggree to behavior

Image-related link engagement level (for the

campaign)

‧

Figure 6.3.37 Image-unrelated link engagement level (for the campaign) To prove the proposition for image-relate link for link building service on engagement site have higher engagement level on engagement site, we develop the hypothesis test (See Figure 6.3.32) and run the two-sample t test to compare:

Figure 6.3.38 Hypothesis of the testing for average engagement level on image-related link and image-unrelated link to target site

Table 6.3.8 Paired-Sample T test for average engagement level on image-related link and image-unrelated link to target site

Paired-Sample Test t df Sig. (2-tailed)

1.31 32 0.1987

The result (Table 6.3.6) shows that the t value is 1.31 and its significance level is 0.1987 (p-value = 0.1987), which is above 0.05. Therefore, the null hypothesis H0

1 Customer's 7 point Likert scale of aggree to behavior

Image-unrelated link engagement level (for the campaign)

𝐻0: 𝜇1= 𝜇2 𝐻1:𝜇1≠ 𝜇2

𝜇1= average enagement level of imegae − related link 𝜇1= average enagement level of imegae − unrelated link

5.12

‧

is not rejected and it means there is not statistically significant evidence to support our proposition that image-related link can generate higher engagement level.

Proposition 6-2.2 For target site (link from inlink) engagement level:

Figure 6.3.39 Image-related link to target site engagement level (for the campaign)

Figure 6.3.40 Image-unrelated link to target site engagement level (for the campaign)

1 Customer's 7 point Likert scale of aggree to behavior

Image-related link to target site engagement level

Customer's 7 point Likert scale of aggree to behavior

Image-unrelated link on target site engagement

level

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

To prove the proposition for image-relate link for link building service on engagement site have higher engagement level on engagement site, we develop the hypothesis test (See Figure 6.3.32) and run the two-sample t test to compare:

Figure 6.3.41 Hypothesis of the testing for average engagement level on image-related link and image-unrelated link

Table 6.3.9 Paired-Sample T test for average engagement level difference in SME-revised and SME-original-choose keywords

Paired-Sample Test t df Sig. (2-tailed)

1.89 32 0.0678

The result (Table 6.3.6) shows that the t value is 1.89 and its significance level is 0.0678 (p-value = 0.0678), which is above 0.05. Therefore, the null hypothesis H0 is not rejected and it means there is not statistically significant evidence to support our proposition that image-related link to target site can generate higher engagement level than image-unrelated link.

Thus in this proposition 6-2, we conclude that image-related link building service though have higher engagement level in average, but fail be statistically supported to have a higher engagement level for image-related link both on inlink site and on target site (link from inlink). Proposition 6-2 will be further dicussed in section

在文檔中品牌合作活動為基礎的顧客參與服務平台:以搜尋引擎優化之觀點 - 政大學術集成 (頁 90-110)