以徵引學術網路來建構學術論文評估指標

國

立

交

通

大

學

資訊學院 數位圖書資訊學程

碩 士 論 文

以徵引學術網路來建構學術論文評估指標

How Construction of the Evaluation Indicators on Scholarly

Literatures Under Citation Network

研 究 生：周永佳

指導教授：孫春在 教授

以徵引學術網路來建構學術論文評估指標

How Construction of the Evaluation Indicators on Scholarly

Literatures Under Citation Network

研 究 生：周永佳 Student：Yong-Jia Zhou

指導教授：孫春在 Advisor：Dr. Chuen-Tsai Sun

國 立 交 通 大 學

資訊學院 數位圖書資訊學程

碩 士 論 文

A Thesis

Submitted to College of Computer Science National Chiao Tung University in partial Fulfillment of the Requirements

for the Degree of Master of Science

in

Digital Library June 2011

Hsinchu, Taiwan, Republic of China

以徵引學術網路來建構學術論文評估

指標

學生：周永佳 指導教授：孫春在 教授

國 立 交 通 大 學 資 訊 學 院 數 位 圖 書 資 訊 學 程 碩 士 班

摘要

大 學 主 要 功 能 為 研 究 與 教 學，大 學 排 名 有 的 是 注 重 人 為 評 分 方 式 ； 有 的 則 是 注 重 科 學 評 比 方 式 。 大 學 排 名 對 形 象 及 知 名 度 有 深 遠 影 響 。 而 學 術 論 文 的 評 鑑 ， 不 僅 可 以 瞭 解 學 術 研 究 結 果 的 差 異 ， 也 可 當 作 選 擇 就 讀 學 校 的 參 考 。 大 學 排 名 評 比 方 式 各 有 不 同 ， 數 據 取 得 來 源 也 不 盡 相 同 。 本 研 究 是 以 大 家 隨 手 可 得 的 免 付 費 學 術 搜 尋 引 擎 G o o g l e S c h o l a r 為 資 料 來 源 。 取 樣 自 國 內 國 立 大 學 之 資 訊 學 院 學 者 ， 從 學 者 的 學 術 論 文 被 引 用 之 角 度，參 考 上 海 交 通 大 學 與 台 灣 高 等 評 鑑 中 心 評 估 指 標 ， 去 建 構 符 合 臺 灣 區 域 性 質 、 學 科 背 景 的 學 術 論 文 評 估 指 標 。 以 學 者 個 人 論 文 被 引 用 情 形 ， 得 出 該 學 院 之 學 術 研 究 能 力 ； 以 不 同 角 度 去 建 構 學 術 論 文 評 估 指 標 ， 再 去 作 排 名 ， 以 見 樹 也 見 林 方 式 ， 來 探 討 它 的 意 義 。

關鍵字：學術評鑑、大學評鑑、論文徵引、Google Scholar

How Construction of the Evaluation Indicators on Scholarly

Literatures Under Citation Network

Student: Yong-Jia Zhou Advisor: Dr. Chuen-Tsai Sun

Degree Program of Computer Science

National Chiao Tung University

ABSTRACT

T he main function of the university is research and teaching. In university ranking, someone pay attention rating in the way artificially, but someone pay attention in scientific.University ranks have far-reaching about image and popularity. It not only merely understands the difference of academic results of research, but also regard as choose the reference that study in the school.

There are many different ways in ranking university, and data source is also different too. The search engine of this research is Google Scholar. It is easily to get and free to everyone.

We use the scholars of computer science of domestic national university as sample. Cited papers of scholars in the angle and consult indicator from Shanghai Chiao Tung University and Higher Education Evaluation and Accreditation Council of Taiwan. We construct evaluation indicators on scholarly literature which meet the nature of Taiwan region and academic background.

It quoted the situation for scholar's personal thesis, and then obtains the academic research ability of this institute. We construct the evaluation indicators on scholarly literatures under citation network with different angles and go to rank. By the way of seeing trees and see the forest too, and probe into its meaning.

Content

ABSTRACT ... IV 1 INTRODUCTION... 1 2 RELATED WORK ... 3 2.1 UNIVERSITY EVALUATION ... 3 2.2 CITATION NETWORK ... 5 2.3 GOOGLE SCHOLAR ... 7

2.4 ACADEMIC RANKINGS USED ... 10

3 THE MODEL ... 14

3.1 DEFINITION OF EVALUATION INDICATORS ... 14

3.2 EQUATION OF MODEL ... 15

3.3 FRAMEWORK OF SYSTEM ... 17

4 EXPERIMENTS ... 19

4.1 DATA ... 19

4.2 DESCRIPTION FOR EXPERIMENT... 20

5 RESULT ... 22

5.1 EVALUATION FORMULA I ... 22

5.2 EVALUATION FORMULA II... 23

5.3 EVALUATION FORMULA III ... 24

5.4 COMPREHENSIVE RESULTS ... 25

List of Tables

Table 1 useful search tips often overlooked by searchers……….…...09

Table 2 academic ranking of world universities index and weight………. 12

Table 3 HEEACT evaluation indicators………...……....13

Table 4 definition of evaluation indicators………...15

Table 5 evaluation formula I...16

Table 6 evaluation formula II………...16

Table 7 evaluation formula III………. ....17

Table 8 objects………..………....19

Table 9 grading of evaluation formula I………...………....22

Table 10 ranking results of evaluation formula I……….…....22

Table 11 grading of evaluation formula II………....23

Table 12 ranking results of evaluation formula II………..…..23

Table 13 grading of evaluation formula III………..24

Table 14ranking results of evaluation formula III……….…..24

Table 15 ranking results of Taiwan university.……...………..34

List of Figures

Figure 1 Citation network………06

Figure 2 Google scholar‟s advanced scholar search………....10

Figure 3 input tem………15

1 INTRODUCTION

University ranks have far-reaching about image and popularity. It is importance for the funds and marketing. There are different kinds of university evaluation： ministry of education official evaluation、schools for the evaluation and private magazine for the evaluation.

University become to market-driven today. It begins to produce quantitative indicators of academic under the pressure of competition. Such as SCI、SSCI、A&HCI, used to measure the technology and social studies into the academic study. The evaluation item of projects and the resulting data will reflect the findings.

Scholar uses to cite others papers and list in reference of his paper. Citation means the article has relationship with literature which it cited. Citation can assess quality of papers and impact of researchers. Citation analysis can construct literature evaluation indictor and rank for university.

Scholars cited others papers or papers cited by others. These phenomena will form academic citation network. Citation each other shows properties of small-world network [1]. The properties are high cluster and low degree of separation.

We propose a research way to use published papers as basis. There are several ways in paper view. First is quantity. Paper whether be cited or not? How much weight? The experimental evaluation indicators reference ARWU and HEEACT.

Why use Google Scholar as data collection platform because it is open access and continue expansion. The scope of key topic includes pdf、word etc.It can evaluate personal or organization. The count space is whole world. Google scholar due to time relationship, it is more complete and objective and respond university capacity better.

This study has several objectives: 1. Study citation network; 2. understand the evaluation indicators; 3. construct indicators of scholarly literatures.

The second section outlines researches made about university evaluation 、citation、 Google scholar and academic rankings used. The third section describes definition of ranking item、equation model and structure of model completely. The fourth section describes detailed data and experiments. The fifth section describes the results of experiments, analyses to the results, and verification of model. The final section uses the structure to discussion its meaning and restriction.

2 RELATED WORK

2.1 UNIVERSITY EVALUATION

In recent years, university evaluation and university rank become domestic and international concerned topics. What is university evaluation? What are the important indicators of scholarly literature evaluation?

The university is no longer the ivory tower specializing in teaching knowledge. It has already made the transition and become knowledge to create and propagate the important strategic place today. It moves towards the new way of community service gradually. The new trend of university development leads to mercerization 、 internationalization and competitiveness. [3]

Since 21st century, various countries have competitively carried on the higher educational reform. Higher education moves towards the direction that the market leads. It causes the high competition of the university and produces the academy evaluation of university.

Our country rose in the university evaluation in recent years. In Taiwan, Ministry of Education carried on the university evaluation in the 92nd year. It adopts SCI, SSCI and EI database, to count total published literature papers of universities as the reference of academy literature evaluation.

The chief of Ministry of Education comments idea of university evaluation: 「Scholastic attainment is the most important of university evaluation. It mainly according the research thesis which the university puts forward in current year as counted in principle, and by issuing it in world...」.

Why will the university need to evaluate? At first, probe into and comment the meaning of university evaluation. Evaluation comment by two ways: quality and

quantity. Scriven who is the famous commenting scholar of U.S.A says:「The purpose of university evaluation is not proving anything, but what is improved」.

Evaluation applies to the university called university evaluation. The purpose is for improving the teaching of university and research level [4]. President of the previous American Stanford University Donald Kennedy once had a section of description catching the spirit to the academic research responsibility of the university. He says the university has academic responsibility to the society; its core is building and constructing the academic activity to the teacher. It is an important reference that amount of research results of scholars and published in famous journals become important reference of promotion and academic evaluation [5].

In 1983, “American news and the world report “(USNWR) begin to offer a rank to the American university. In 2003, Shanghai Communications University World University academy rank (ARWU) quoted and ranked university. The number of Nobel Prize obtained by scholars and their publications indexed in SSCI/SCI. Since 2007 in Taiwan, Higher Education Evaluation & Accreditation Council of Taiwan (HEEACT) in higher education analyzes the quality of scholar‟s publications every year.

Academic evaluation can be said that it is the first task for improving research ability for all academic programs. It is also to identify the current location of the university and advantages of the necessary measures. By science technical evaluation, It is a guiding role in a positive face on the national academic policy development, support and reward the choice of the direction of peer surface. It can understand the current performance, with the power to remind and alert effect in the negative side [6].

Academic evaluation can be taken by two ways: peer review and citations. Peer review is a qualitative evaluation of experts, policy makers can be defined as experts

from relevant fields to seek proposals were judged based on expert advice to improve the existing problems and missing. The limitations of peer review discussed often include paper review results depend on a small number of examiners, senior researchers cannot fully understand the new research. Besides, it spent a lot of time and money. At this point, the objective and quantitative measurement of citation information was faithfully [7].

In general, frequent use of evaluation indicators of assessment: the total number of papers、 total cited papers、 the number of average citations per paper、 cited a specific number of times greater than the total value of the paper etc[8].

There are often regarded as the basic "quantity "and "quality" indicators to evaluate academic universities: the amount of papers, paper citations, the paper cited the average number of journals published in the influence and so on. Besides, it is more able to demonstrate academic evaluation indicator by the amount of “excellent literature such as highly cited papers, popular articles and top post.

2.2 CITATION NETWORK

Egghe & Rousseau [11] explain “when a document di cites a document dj, we can

show this by an arrow going from the node representing di to the document

representing dj. In this way the documents from a collection D form a directed graph,

which is called “citation network”. Figure 1. displays such a network,

It illustrates the reference connections between nineteen articles published on the subject between 1954 and 1970. In Figure 1. The network was constructed using the

Figure 1. Citation network

Citation analysis is a measured way of information, commonly used for assessment of journals and researchers.

Citation analysis originated in the 1963 .Scientific American information companies (Institute for Scientific Information, referred to as ISI) selected journals and reference journals to establish citation database. It is the "Science Citation Index"(Science Citation Index, referred to as SCI). Because it is including the citation information of journal papers, so SCI becomes the scientific literature important tool.

Then the company ISI has prepared Social Science Citation Index (SSCI) and Journal Citation Reports (JCR). SSCI expands the assessment scope of social science journals.

R.N. Kostoff says that citation analysis measured in research evaluation has several functions :(1) Citation can provide relevant information to this paper. (2) References to articles and other references in this paper for the links can provide the contributions of this article. (3)References can offer radioactive tracing of the paper effect and the link tracking. (4)Citation can provide information of other studies, including what papers in academic field. [13]

The frequently cited paper is greater value than seldom cited paper. A literature has been referenced is more contribution than un-cited literature [14]. The higher frequency of being cited author means he (she) has high-impact.

Each paper and other papers connected by reference. Scholars cite literature mutually because they are in the same research area. It shows properties of small -world network: high cluster. Papers cited by the same research area means that it is professional.

The weight of paper is different that cite by domestic or foreigner. The paper cited by other countries means it has high visibility, so it has higher weight.

Citations can occur without the authors knowing each other and can span across time. [15]

2.3 GOOGLE SCHOLAR

2.3.1 Google Scholar significance of the times

Internet network developed, making data collection without time and space restrictions, and complete information collected off than before. It can better reflect the research capacity of schools.

Google is the search engine, for the toll-free, high market share, with strong capabilities of search. The advantages of Google Scholar will use Google search resources in the academic and thus become the world famous Internet references cited by tool.

2.3.2 Google Scholar characteristics

Google Scholar provides a simple way to search for scholarly literature extensively, including resources from multiple channels, such as academic publishers,

professional societies, universities and other academic institutions have. Indexed content includes peer-reviewed reports, preprints, papers, books, abstracts and articles and so on [19].

Sorted search results in accordance with the level of the number of references, the literature most cited more front row. It can not only find references but also be founded in the literature. It is sorted according to their degree of correlation. Google Scholar allows us to research from the academic world to find the most relevant studies.

2.3.3 Key advantages and capabilities of Google Scholar

Google Scholar search interface is a simple search box. A variety of search strategies are available keywords, author, and title of the article so as searches. It can also be compound search [20].

Command Function Example

+ Searches stop words +to +be +or not +to +be - Removes a word or phrase eyes diseases –animal OR Boolean operator to

expand search. Must be capitalized.

bibliometrics OR informetrics

“Quotation marks” Phrase searching “Persian Gulf War” intitle: Returns results that include

the search term in the title of the document/page.

intitle:competitive intelligence allintitle: Searches for all the words

in the title of the page

allintitle:competitive intelligence

site: Searches for the word in the site/domain name. Limits searches to a special domain or site.

site:ac.uk

“digital libraries” site:edu

inurl: Searches for the word in the URL

inurl:webdex

in the URL

author: Searches for the word in the author‟s name

author:Berners-lee filetype: Limits file type and

retrieves a special file format

metadata filetype:pdf

* Searches the phrase (enclosed in quotation marks) and * replaced by any single word. This operator can be used for proximity searching to retrieve a compound name or a phrase that appear a specified number of words in the middle of it.

“web * analysis” “citation ** analysis” “web *** ontology” “Anglo American * Rules”

.. Number range “digital camera” “5..5000 megapixel”

Table 1 useful search tips often overlooked by searchers

2.3.4 Google Scholar’s Advanced Scholar Search

It makes possible to limit results to a date range or specific publication or journal by using “Advanced Scholar Search” (see Figure2).

Figure2 Google scholar‟s advanced scholar search

2.4 ACADEMIC RANKINGS USED

2.4.1 U.S.News & Report

The new 2010 World's Best Universities rankings are based on data from the QS World University Rankings [23], which were produced in association with QS Quacquarelli Symonds. QS Quacquarelli Symonds, one of the world's leading networks for careers and education has been publishing world rankings since 2004.

The 2010 listing also includes the Top 100 global rankings in the fields of arts and humanities; engineering and IT; life sciences and biomedicine; natural sciences; and

social sciences.

U.S. News and World Report's ranking criteria include peer assessment, the University's academic reputation, graduation situation, the acceptance rate of students, financial resources, alumni donation rate.

.

2.4.2 ARWU

Shanghai Jiao Tung University world-class university research centers and Higher Education Research Institute published the Academic Ranking of World Universities (ARWU) online since 2003[25]. They announce and update ranked results in the network every year. The distinguishing feature is that pay attention on top academic research. Such as Number of fellowships about the Nobel Prize or the Fields Medal (Fields Medal, for excellence awards, established a young mathematician); the number of articles published in the well-known journal "Nature" and "science".

The number of papers record in the Science Citation Index (SCIE) and Social Science Citation Index (SSCI).

The advantage of ARWU ranking method is the objectivity and transparency. It can not reflect the research results of most Professors. Disadvantage is that other indicators like the Nobel Prize, only a handful of winners.

Level indicators

Secondary indicators Code Weight The quality

of education

Won a Nobel Prize and Fields Medal equivalent to the number of alumni

Alumni 10% Teacher

Quality

Won the Nobel Prize and Fields Medal equivalent number of teachers

Award 20% The highest number of disciplines cited the

number of teachers

HiCi 20% Research In the " Nature "and" Science "equivalent to

the number of published papers

By Science Citation Index ( SCIE ) and Social Science Citation Index ( SSCI ) the number of papers included

PUB 20%

Teachers were shown to

Division of the five index scores mean PCP 10%

Table2 academic ranking of world universities index and weight

2.4.3 HEEACT

HEEACT (Higher Education Evaluation and Accreditation Council of Taiwan ) cooperate with the National Taiwan University analysis quality and quantity of 1998-2008 papers, comparing the university academic competitiveness [26].

The advantage is that rating of the university is based on the quantitative data. Ranking method is more objective and rational. The disadvantage is that rank university based on the most narrow concept research paper by the paper quality and quantity.

Dimensions 2009 targets in the field, regardless of Weight Academic productivity Number of papers nearly a year (1998-2008) 10 20 Number of papers that year (2008) 10 Academic influence Number of papers are cited nearly a year (1998-2008)

10 30

the number of papers cited (2007-2008) Papers nearly a year the average number of citations

(1998-2008)

10

Academic excellence

Nearly two years of h index (2007-2008) 20 50 Number of highly cited papers (1998-2008) 15 Number of high-impact journal articles (2008) 15

3 The Model

Because of rapid changes in information field, the experiment analysis scholars‟ papers from National University of Computer Science in recent 5 years (2006-2010).

Evaluation indicators modified the indicators used by 「 Higher Education Evaluation & Accreditation Council of Taiwan 」 and 「 Shanghai Chiao Tung University」. From ARWU, we reference the evaluation indicator is scholar‟ paper has been included on SCI or EI or SSCI or A&HCI as academic excellence indicator. From HEEACT, we reference number of papers (2006 to 2010) as academic productivity indicator; we reference the journal paper has been cited as academic influence indicator; we reference the journal paper is highly cited as academic excellence indicator. The weights of those evaluation indicators base on evaluation formulas.

We use Google Scholar to analysis data. The model is details described below.

3.1 DEFINITION OF EVALUATION INDICATORS

Level

indicators Secondary indicators Code Academic

Influence Scholar‟ paper has been cited 1 time (or more) Cited

Academic Excellence

Scholar‟ paper is cited more than 100 times

(inclusive) HighCite

Scholar‟ paper has been included on SCI or EI or

SSCI or A&HCI PUB

Academic Popularity

Scholar‟ paper is cited by non-Taiwanese Foreign Scholar‟ paper is cited by Taiwanese Domestic

Academic

Professional Scholar‟ paper is cited by Computer Science SameArea Academic

Productivity Numbers of papers (2006-2010) Qty Table4 definition of evaluation indicators

Cited

Scholars’ papers of National University of Computer Science

Ranking

HighCite PUB Foreigner Domestic SameArea Qty

Figure3. input item

3.2 EQUATION OF MODEL

Level

indicators Secondary indicators Code Weight Academic

Professional Scholar‟ paper is cited by Computer Science SameArea 20% Academic

Popularity

Scholar‟ paper is cited by Taiwanese Domestic 20% Scholar‟ paper is cited by non-Taiwanese

Academic Excellence

Scholar‟ paper is cited more than 100 times

(inclusive) HighCite 30%

Table5 evaluation formula I

The purpose of evaluation formula I is to construct literature evaluation indictor from 3 ways: professional, popularity and excellence.

Evaluation formula I is more globalization that the paper not only cited by Taiwanese but also cited by foreigners. Popularity of them is 50%.

Level

indicators Secondary indicators Code Weight Academic

Professional Scholar‟ paper is cited by Computer Science SameArea 25% Academic

Popularity Scholar‟ paper is cited by Taiwanese Domestic 25% Academic

Influence scholar‟ paper has been cited 1 time (or more) Cited 25% Academic

Productivity Numbers of papers published Qty 25% Table6 evaluation formula II

The purpose of evaluation formula II is to construct literature evaluation indictor from 4 ways: professional, popularity, influence and productivity. Each indicator is 25%.

Evaluation formula II is more localized that the paper only cited in Taiwan.

Level

indicators Secondary indicators Code Weight Academic

Excellence

Scholar‟ paper has been included on SCI or EI

Scholar‟ paper is cited more than 100 times

(inclusive) HighCite 30%

Academic Popularity

Scholar‟ paper is cited by Taiwanese Domestic 15% Scholar‟ paper is cited by non-Taiwanese Foreign 25%

Table7 evaluation formula III

The purpose of evaluation formula III is to construct literature evaluation indictor from excellent and popularity. The paper cited by foreigners is higher weight than cited by Taiwanese.

Evaluation formula III emphasize academic excellence. Excellence of them is 60%

3.3 FRAMEWORK OF SYSTEM

INTERNET Google SCHOLAR PROCESS Evaluation formula I Evaluation formula II Evaluation formula III DB FOR RANKING DATA

Figure4 the system framework

We choose Google scholar to be data collection platform because it is open-access and always continue expansion. Besides, the count space is whole world.

In this research, we collect scholars‟ papers from the National University of Computer Science published in 2006-2010. By use Google Scholar to collect these data through Internet, according to different evaluation formula to generate different evaluation indicators on scholarly literatures.

4 EXPERIMENTS

4.1 DATA

The experiment chooses seven college of computer science of national university. They are listed below.

School name Abbreviation

National Taiwan University NTU

National Tsing Hua University NTHU National Cheng Kung University NCKU National Chiao Tung University NCTU National Taiwan University of Science and Technology NTUST National Central University NCU National Taipei University of Technology NTUT

Table8 objects

In our research，the school we choose famous universities（National Taiwan University、National Tsing Hua University、National Chiao Tung University） and technology universities（National Taiwan University of Science and Technology、 National Taipei University of Technology） and southern university（National Cheng Kung University）and northern university（National Central University）. We got scholars‟ papers of seven college of computer science of national university from the National Science Council and filter out papers published from 2006 to 2010 to be our database. The amounts of data that we picked up are 3106. And we have 359 scholars for the data; and the paper belonged to SCI、EI、SSCI、A&HCI to be our database . We used Google scholar to get cited information. Those cited data are also

4.2 DESCRIPTION FOR EXPERIMENT

Google scholar is more structured and free charge. We send a query string about scholar‟s paper to Google scholar and it return relative citation information.

Has the paper been cited? Is the paper cited by same field? Whether the paper cited by natives? Whether the paper cited by foreigners? According these data produce rating method. Scoring is as follows.

(A) Belonged to SCI/EI/SSCI: as long as there is belonged to SCI etc. that are one point.

(B) The number of citation: citation number is more than one (inclusive) that is one point.

(C) High number of citations: The citation numbers are over than 100 (including), that is one point.

(D) Citation by same field: as long as one citation by the same area, that is one point. (E) Citations by nationals: as long as one citation by native, that is one point.

(F) Citation by foreigners: as long as one citation by foreigner, that is one point. The formula based on the score for each evaluation index add, then obtained scores of these schools and go to rank.

How to obtain information of experiment? The step described as follows.

First, we get scholars and journal papers of the seven universities related computer science department from the web site of National Science Council.

Second, in journal paper, it is marked belong to SCI, EI, SSCI, A&HCI information.

Third, in Google scholar, we input name of journal paper and get cited information. The number of citations will be to determine whether the journal paper has been cited and whether the cited count more than 100 or not.

Fourth, we use “search within articles citing” function of Google scholar and enter keyword “computer science". It is to determine whether the journal paper cited by same area or not.

Fifth, we use “search within articles citing” function of Google scholar and enter keyword “Taiwan”、“ROC”、“republic of china”. It is to determine whether the journal paper cited by domestic (Taiwanese) or not.

Sixth, we use “search within articles citing” function of Google scholar and enter keyword “Taiwan”、“ROC”、“republic of china”. If it not founds cited information, then the journal paper cited by foreigners (non-Taiwanese).It is to determine whether the journal paper cited by domestic (Taiwanese) or not.

Seventh, we use “search within articles citing” function of Google scholar and enter keyword “Taiwan”、“ROC”、“republic of china”. If it founds cited information, then enter keyword “USA”etc. It is to determine whether the journal paper both cited by domestic (Taiwanese) and foreigners (non-Taiwanese).

Overall, we obtain journal paper form National Science Council first. Then we use Google scholar to get cited information of journal paper.

5 RESULT

5.1 EVALUATION FORMULA I

Grading of evaluation formula I is as follows.

Code SameArea Domestic Foreign HighCite

Weight 20% 20% 30% 30%

Table 9 grading of evaluation formula I Scoring result of evaluation formula I is as follows.

University SameArea Domestic Foreign HighCite Total NCKU 0.578366446 0.42384106 0.472406181 0.002207506 0.342826 NTU 0.550247117 0.403624382 0.45785124 0.013179572 0.332084 NCTU 0.557180851 0.417553191 0.439414115 0.003989362 0.327968 NCU 0.505535055 0.361623616 0.42804428 0.007380074 0.304059 NTUST 0.519685039 0.362204724 0.421259843 0 0.302756 NTHU 0.454411765 0.308823529 0.375 0.001470588 0.265588 NTUT 0.348314607 0.258426966 0.247191011 0 0.195506

Table 10 ranking results of evaluation formula I

We normalize “evaluation formula I” based on journal paper of the computer science of the university.

The purpose of evaluation formula I is to construct literature evaluation indictor from 3 ways: professional, popularity and excellence.

The journal paper cited by the same area frequently means the journal paper is professional and it shows high cluster of small-world network.

Evaluation formula I is more globalization that the paper not only cited by Taiwanese but also cited by foreigners. Popularity of them is 50%.

From academic excellence view, there are five universities that the journal paper has cited more than 100 times.

In evaluation formula I, number one is NCKU, number two is NTU and number three is NCTU.

5.2 EVALUATION FORMULA II

Grading of evaluation formula II is as follows.

Code SameArea Domestic Cited Qty

Weight 25% 25% 25% 25%

Table 11 grading of evaluation formula II Scoring result of evaluation formula II is as follows.

University SameArea Domestic Cited Qty Total NCTU 0.555851064 0.417553191 0.558510638 0.242112 0.443507 NCKU 0.578366446 0.42384106 0.578366446 0.145847 0.431605 NTU 0.550247117 0.403624382 0.561779242 0.195428 0.42777 NTUST 0.519685039 0.362204724 0.523622047 0.081777 0.371822 NCU 0.505535055 0.361623616 0.516605166 0.08725 0.367754 NTHU 0.454411765 0.308823529 0.457352941 0.218931 0.35988 NTUT 0.348314607 0.258426966 0.348314607 0.028654 0.245928

Table 12 ranking results of evaluation formula II

The first three indicators, we normalize “evaluation formula II” based on journal paper of the computer science of the university. The fourth indicator we normalize on journal paper of the computer science of seven universities.

The purpose of evaluation formula II is to construct literature evaluation indictor from 4 ways: professional, popularity, influence and productivity. Each indicator is 25%.Evaluation formula II is more localized that the paper only cited in Taiwan.

From academic productivity view, NCTU published maximum quantity of journal paper.

From academic influence view, journal papers of NCKU are most popular references.

From academic popularity view, journal papers of NCKU are most cited by Taiwanese.

From academic professional view, journal papers of NCKU are most cited by same area (computer science field).

In evaluation formula II, number one is NCTU, number two is NCKU and number three is NTU.

5.3 EVALUATION FORMULA III

Grading of evaluation formula III is as follows.

Code Pub HighCite Domestic Foreign

Weight 30% 30% 15% 25%

Table 13 grading of evaluation formula III Scoring result of evaluation formula III is as follows.

University Pub HighCite Domestic Foreign Total NCKU 0.818984547 0.002207506 0.42384106 0.472406181 0.428035 NCU 0.763837638 0.007380074 0.361623616 0.42804428 0.39262 NCTU 0.699468085 0.003989362 0.417553191 0.439414115 0.383524 NTUST 0.744094488 0 0.362204724 0.421259843 0.382874 NTU 0.512355848 0.013179572 0.403624382 0.45785124 0.332667 NTUT 0.617977528 0 0.258426966 0.247191011 0.285955 NTHU 0.447058824 0.001470588 0.308823529 0.375 0.274632

Table 14 ranking results of evaluation formula III

We normalize “evaluation formula III” based on journal paper of the computer science of the university.

The purpose of evaluation formula III is to construct literature evaluation indictor from excellent and popularity. The paper cited by foreigners is higher weight than cited by Taiwanese.

Evaluation formula III emphasize academic excellence. Excellence of them is 60%.

From academic excellence view, the journal paper of NCKU is most published in SCI etc.Besides,NTU is most high-cited.

From academic popularity view, journal papers of NCKU are most cited by natives (Taiwanese) and foreigners (non-Taiwanese).

In evaluation formula III, number one is NCKU, number two is NCU and number three is NCTU.

5.4 COMPREHENSIVE RESULTS

There are different ranking results by choosing various indicators and various

weights.

Comprehensive results of the three evaluations formula, there are different ranking results in the experiment.

Evaluation formula I emphasized academic popularity. The evaluation indicator emphasized worldwide. The top three is NCKU、NTU and NCTU.

Evaluation formula II emphasized localized. The top three is NCTU、NCKU and NTU.

Evaluation formula III emphasized academic excellence. The top three is NCKU、 NCU and NCTU.

6 CONCLUSION

The primary aim of our model is to find what and how to construct evaluation indicators on scholarly literatures under citation network.

We use the journal paper of the National University of computer science to be analyzed. Today, Internet developed; we can easily get any information through Internet platform. In this research, we obtain journal paper information of our experiment from the National Science council web site. Besides, we use Google scholar to get cited related information of our experiment because Google scholar is a powerful search engine and free. We reference ARWU and HEEACT rank system to construct academic evaluation indicator model. We successfully produced the evaluation indicators on scholarly literatures under citation network.

In this research, every indicator has its expression in a target-oriented. We try to present a variety of academic indicators from multiple aspects. We also explain careful the significance of each indicator.

It can be developed as point of view to construction of academic indicators by scholar‟s papers under citation network. But it may not apply to papers published less. This study currently only completed under Computer Science of National University of Republic of China. It can continue to study other areas in the future.

REFERENCE

[1] D. J. Watts and S. H. Strogatz, Collective dynamics of „small-world‟ networks. Nature 393, 440–442 (1998).

[2] Watts, D. J. Small Worlds, The Dynamics of Networks Between Order and Randomness .Princeton Univ. Press, Princeton, NJ (1999).

[3]Mei-Ling Wang. University Research Assessment and Citation Analysis. Journal of Library and Information Science 31（1）：5 – 13(Apr., 2005)

[4]李聰明，大學評鑑的研究(臺北：中國文化大學出版部，民 86 年)，頁 13。 [5] Donald Kennedy 著，楊振高譯，學術這一行(臺北：天下，民 89 年)，頁 2。 [6]Huang, M. H., Chang, H. W., & Chen, D.Z. Research evaluation of research-oriented universities in Taiwan from 1993 to 2003. Scientometrics, 67(3), 419-435(2006).

[7] Mu-Hsuan Huang. Application of H-index for Research Evaluation at University Level. 高教評鑑 1:2(Feb 2008)

[8]Hirsch, J. E. (2005). An index to quantify an individual‟s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569-16572.

[9]Ball, P. (2005).Index aims for fair ranking of scientists. Nature, 436,900.

[10] Hirsch, J. E. (2005). An index to quantify an individual‟s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569-16572.

[11]Egghe, L. & Rousseau, R. (1990). Introduction to Informetrics: Quantitative

Methods in Library, Documentation and Information Science. Amsterdam, NL:

in the anaesthetics field: „Perturbation of ion transport‟. British Journal of

Anaesthesia, 43: 814

[13] R.N.Kostoff, “The use and misuse of citation analysis in research evaluation”, Scientometrics, 43:1(1998):27-29

[14] A.T.Balaban,”How should citations to articles in high- and low-impact journals be evaluated, or what is a citation worth?” Scientometrics, 37:3(1996):495

[15] S. Redner. How popular is your paper? an empirical study of the citation distribution. The European Physical Journal B, 4(2):131–134, 1998.

[16] How can we investigate citation behavior? A study of reasons for citing literature in communication: Journal of the American society for information 51(7):635-645, 2000

[17] 19.Vaughan, L.; Shaw, D. (2008) A new look at evidence of scholarly citations in citation indexes and from web sources, Scientometrics, vol. 74, no. 2. pp. 317-330.

[18]Smith, Linda C., "Citation Analysis," Library Trends 30:1(Summer 1981): 94-98.

[19] Google Scholar，http://scholar.google.com

[20]Google Scholar: The New Generation of Citation Indexes Libri, 2005, vol. 55, pp. 170–180

[21]Kousha, K.; Thelwall, M. (2007) Google Scholar Citations and Google Web/URL Citations: A Multi-Discipline Exploratory Analysis, Journal of the American Society

for Information Science and Technology, vol. 58, no. 7, pp. 1055-1065.

[22] Jacsó, P. (2005) Google Scholar: the pros and the cons, Online Information

Review, vol. 29, no. 2, pp. 208-214.

[23] http://www.usnews.com/rankings

[24]http://www.usnews.com/education/worlds-best-universities/articles/2010/09/21/w orlds-best-universities-top-400-?PageNr=8

[25] http://www.arwu.org/Chinese/ARWUMethodology2009.jsp [26] http://ranking.heeact.edu.tw

[27]http://www.topuniversities.com/university-rankings/world-university-rankings/me thodology/simple-overview

附錄 A SMALL-WORLD NETWORK

The whole world is composed of many different individuals. If there is a relationship between them, there is a link between each other. These links are interwoven into the relationship between different forms of network, called the Social Network. The branching degree (vertex degree) is the link between the individual and the number of individuals. For example, 4 to the branch level, means the individual X and individual Y1, Y2, Y3, Y4 are linked. The most common way to link social networks based on different links: regular Network, small-world network and random Network.

Small-world network in 1998, Cornell University Duncan J. Watts and Steven thesis advisor Strogatz co-sponsored paper, "Collective dynamics of small-world network", opened a small world network of trend. Small-world network is neither completely ordered network, nor entirely random network, but somewhere between the two networks. Small-world network is a network in order adds a shortcut on the random (Shortcut). It is the current closest to the real social network.

Two major characteristics of social networks are clustering and degree of separation. Degree of clustering is connected to the individual and the extent of the individual neighboring. Degree of separation is the shortest number of links will have to go through the intersection. The small world network has ordered a high degree of clustering network (Highly clustered) and random network of low degree of separation (small characteristic path length) features. The most famous small-world phenomenon is between two people of any irrelevant, can be linked together by six degrees of a relationship called Six Separated.

In the real world, there is a wide range of small world network, such as the western United States electricity supply system, the film Actors map, road map, links

to networks of human neurons, modes of spread of infectious diseases, are showing a small world network phenomenon.

附錄 B 資料

完整資料敘述 本文以國內國立大學資訊學院學者教授從 2006 年到 2010 年發表的論文為基 礎，參考財團法人高等教育評鑑中心基金會及上海交通大學排名系統所用的評鑑 指標，試圖建立學術論文評估指標，探討在小世界網路理論為基礎的徵引學術研 究下建構學術論文評估指標可行性及方式，並產生排名供選擇就讀或就業學校的 參考。 本研究所使用的資料為國立台灣大學電機資訊學院資訊相關系所、國立清華 大學電機資訊學院資訊相關系所、國立成功大學電機資訊學院資訊相關系所、國 立交通大學資訊學院、國立台灣科技大學電資學院資訊相關系所、國立中央大學 資訊電機學院資訊相關系所、國立台北科技大學電資學院資訊相關系所，共七所 學校。為了盡量使用完整而可供研究的資料，本研究主要的分析對象是從 2006 年到 2010 年上述國內國立大學資訊相關系所學者教授所發表的論文，共 5 個年 度，資料包含下列欄位：1. 該篇論文所屬學院；2.論文發表年度；3. 該篇論文 學者教授名字；4. 該篇論文名稱；5. 該篇論文是否收錄在 SCI/EI/SSCI；6. 該 篇論文被引用次數；7. 該篇論文是否曾被引用；8. 該篇論文引用次數是否超過 100 次(含)；9. 該篇論文是否被同領域引用；10. 該篇論文是否被本國人引用； 11. 該篇論文是否被外國人引用。我們將這些資料彙整成以該篇論文所屬學院為 主要索引依據，共有 359 位學者，3106 筆資料，作為建構學術論文評估指標的 輸入資料。

附錄 C FAMOUS RANKED SYSTEM

There are different ranking results by choosing various indicators and various weights. In famous ranked system, HEEACT emphasized long-term and recent research performance; ARWU emphasized top academic performance; THE and QS emphasized peer review.

The following chart is ranking result from the world's famous ranked system. University HEEACT ARWU THE QS

NTU 114（1） 127（1） 115（2） 94（1） NCKU 302（2） 256（2） - 283（3） NTHU 346（3） 314 107（1） 196（2） NCTU 479 313（3） 181 327 NCU - 443 - 398 NTUT - - - - NTUST - - - 370

Table15 ranking results of Taiwan university

Ps. - : That did not enter the ranks; （）: Figures in the domestic ranked among top three