Evaluating the educational outcome gains of students enrolled in residential colleges: An application of the block design

(1)

Evaluating the educational outcome gains of

students enrolled in residential colleges: An

application of the block design

Article · July 2017 DOI: 10.5861/ijrse.2016.1628 CITATIONS

0

READS

44

4 authors, including: Some of the authors of this publication are also working on these related projects: Analyzing the factors that promote international students' loyalty and positive WOM

View project Hung Chao Hsiang National Chengchi University 5 PUBLICATIONS 6 CITATIONS SEE PROFILE Gregory S Ching Fu Jen Catholic University 84 PUBLICATIONS 194 CITATIONS SEE PROFILE

(2)

Evaluating the educational outcome gains of students

enrolled in residential colleges: An application of the block

design

Hu, Yueh-Luen

Department of Education, National Chengchi University, Taiwan (joyhu@nccu.edu.tw)

Hung, Chao-Hsiang

Department of Education, National Chengchi University, Taiwan (aka0518@gmail.com)

Ching, Gregory S.

Graduate Institute of Educational Leadership and Development, Fu Jen Catholic University, Taiwan (gregory_ching@yahoo.com; 094478@mail.fju.edu.tw)

Hung, Guang-Chu

Department of Business Administration, Shih Hsin University, Taiwan (gchung@cc.shu.edu.tw)

Received: 3 July 2016 Revised: 20 September 2016 Accepted: 26 September 2016

Available Online: 27 September 2016 DOI: 10.5861/ijrse.2016.1628

ISSN: 2243-7703 Online ISSN: 2243-7711

OPEN ACCESS

Abstract

The rebirth of the residential college system of education in Taiwan is already at its peak with

several universities already immersed in the new learning design for quite some time. To

evaluate its performance, the theoretical concept behind student engagement is adapted as a

basis of analysis. However, in order to effectively determine the value-adding effects of

undergoing a residential type of system some statistical considerations must be acknowledge.

Therefore, in order to compare the educational outcome gains of students who studied under

the residential college with non-residential (ordinary) students, the current study considers the

factor school as a block variable, while the residential college as treatment variable. Data

collected are from 536 residential college and 823 non-residential students enrolled at four

key universities that practices the residential college learning design from school year 2014 to

2015. Statistical analysis utilizing the block design results show that there exist a significant

difference between the residential and non-residential college students’ practical and social

competencies, while no significant differences were found on the students’ general

competencies. In essence, the use of a block analysis denotes that residential colleges are

quite unique and contextual in nature, which focuses not only on domain specific knowledge,

but more importantly on diverse skills including nationalism and active participation. Current

findings should be able to help higher education institutions design better course programs

and activities that promote student-faculty interaction towards more holistic and meaningful

student engagement.

Keywords: student engagement; residential colleges; educational outcomes; block design;

(3)

Hu, Y.-L., Hung, C.-H., Ching, G. S., & Hung, G.-C.

Evaluating the educational outcome gains of students enrolled in residential colleges: An

application of the block design

1. Introduction

For the past few years, a revival is seen on the concepts of residential colleges in Taiwan (Hu, Ching, & Hung, 2015). Similar to that of the United States (US), residential colleges are said to composed of decentralized academic societies or associations composed of faculty and student members (Duke, 1996). Considering a report from Harvard University (2010), which reiterated that the role of general education to include the concept of residential houses, wherein it is said to have contributed to the overall intellectual, ethical, and personal growth of undergraduates. Moreover, the report stresses that education should have the essence of holistic learning; it ought to cultivate students that are able to comprehend life education, critical thinking, civic participation, as well as to understand the notion of self-reflection. These actually portrayed many key similarities within educational outcomes that are also reflected on Taiwan’s residential colleges (Hu, Ching et al., 2015), including some other universities in Taiwan (Chen, 2014; Chen & Ching, 2012).

Residential college actually builds on the notion that new students entering their university years should feel welcomed by senior students and faculty. As the years progresses, their relationships and activities should help them develop a sense of belongingness to the institution and ownership of its collective values. A process that affect the students’ beliefs, attitudes, and knowledge (Astin, 1977). Being reinforced by living and learning together (Ryan, 1992), residential colleges is seen as a catalyst for productive student-faculty interactions (Cox & Orehovec, 2007; Hu, Hung, & Ching, 2015). Within residential colleges, students are able to grow and help their peers within their academic, social, and/or personal development (Durrani & Khan, 2009). Typically, residential colleges range in size from 250 to 500 members. A distinction is that students from different colleges and departments are house together for the entire duration of their studies, however, this sometime differs from institutions to institutions, wherein duration of the residential college are either only during their freshmen and/or sophomore years. Similar within residential colleges is the accompaniment of senior staffs comprising of faculty, counselors, and sometimes members of the local community. Emphasis is placed on achieving a careful balance of academic interests and backgrounds present in the university as a whole. In essence, residential colleges are somewhat smaller versions of their universities (Smith, 1994).

It is no doubt that emphasizing on the residential college system is not merely blindly following the trend of top universities, but it is the core ideas needed in the development of a university’s general education program. Residential colleges have actually evolved over the centuries and gradually influenced by their different local contextual conditions (such examples can be seen in both United Kingdom and the US). In effect, residential colleges in some sense encompasses various terms that are used interchangeably (Pike, 1999). Following this line of argument, the current study is focused primarily on evaluating the various educational outcome gains on students undergoing the residential college system. Furthermore, framework of evaluation shall anchored on the previous findings of Hu and her colleagues (2015), which noted the concepts of student engagement within the residential college system in Taiwan.

2. Residential colleges in Taiwan

With the current predicament of low number of incoming students in Taiwan (Ching, Lien, & Chao, 2014; Chou & Ching, 2015), hence, in order to attract more enrollees, a drive in creating a unique institutional competitive advantage is a must. One distinct program is the establishment of residential colleges. It is suggested that the creation of a residential college culture, educational atmosphere within the university dormitory are enhanced. Researchers also noted that residential colleges are aimed to elaborate the peer-to-peer

(4)

(Duke, 1996; Gary, Charles, & Thomas, 1997; Ryan, 1992). Although residential colleges do not necessarily improve academic achievement, however, it can make freshmen adapt more quickly to university life. In addition, it also improves peer-to-peer social interaction, and eventually enhances students’ engagement.

For school administrators in Taiwan, residential college seems to provide students with a framework for successful learning, more specifically is on emphasizing the unique learning needs of individual students, while making them feel at home and comfortable. In sum, as residential college emphasizes more on life education, peer learning, and other non-academic learning, institutional uniqueness together with its founding philosophies are passed on. In Taiwan, several higher education institutions such as: National Tsing Hua University (NTHU), National Chung Cheng University (NCCU), National Chengchi University (NCU), and Tunghai University (TU) have decided to use the residential college model in developing their general education programs. Various information regarding their residential colleges are hereby summarized, particularly the year of establishment, scales, core concepts, curriculum, and other specific characteristics and strategies that are being employed (Note: Information presented below are gathered and collected within the schools’ respective websites. See Table 1).

This section integrates the various experimental implementations of residential colleges in Taiwan: Tsing

Hua College for NTHU, CCU Elite Program for NCCU, Chengchi College Project for NCU, and Liberal Arts

College for TU.

School Type - One of the objectives of this study is to determine the effectiveness of the establishment

of residential colleges in Taiwan. More specifically, focusing on the difference between the fields of Science and non-Science institutions, such as the NTHU and NCCU are more focused on the Sciences, while NCU and TU are both known for their social sciences (non-Science) programs. (Note: Analysis can be found in the later section of this paper.)

College Scale - The number of college students that have been accumulated since the implementation

of each of the residential colleges since its year of establishment. As different residential colleges differ in form and size, hence, each distinct higher education institutions have different number of students in their residential college program. For instance, the NCU started to implement their residential college model in 2008; activities such as a freshman orientation camp or tutor system is basically targeted to all freshmen student and is also open to senior students. A total of around 2000 students attended the orientation camp; therefore, NCU has the most number of participants among the four universities mentioned in this study. Other universities such as the NTHU, TU, and NCCU are selecting college students to participate in their respective residential college program. Each school year, recruitment of new student participants is established, wherein only a limited number of slots are available; however, the selection system seems to get more and more competitive each year with signs of increasing number of applicant. Tsing Hua College recruits 150 student participants each school year for the past three years, hence, a total of 450 student participants are included in their current residential college program. The CCU Elite Program recruits 80 student participants each school year for the past three years, however, participation is of the voluntary basis, therefore, a total of around 200 student participants are currently included in their residential college program. Lastly, the Liberal

Arts College recruits around 100 student participants each school year since 2008, thus, a total of around 300 student participants are currently in their residential college program (data collected as of 2014).

Founding Concepts and Specific Goals - This section integrates the Taiwan residential colleges

founding concepts and goals. Much similar to the concept of Living-Learning Communities (LLC) of the US; LLC are designed to create the intimate climate of a small college within large universities (National Study of Living-Learning Programs, 2007). Residential colleges in Taiwan also strongly emphasize the personal development through engagement with the broader community, which is similar to the LLC overall goals (Shapiro & Levine, 1999; Wawrzynski & Jessup-Anger, 2010;

(5)

Wawrzynski, Jessup-Anger, Stolz, Helman, & Beaulieu, 2009). Among the four residential colleges in Taiwan, the most prominent goal is to develop the students’ life education that is embedded within the general education program. In addition, institutions also added their distinctiveness in lieu of their history and founding goals. In such a way that each of the residential colleges are both different, but also similar in some ways. In essence, residential colleges in Taiwan generally fosters the students’ development through extra-curricular activities, which are also said to be educationally purposeful, hence, quite similar to the concept of student engagement as proposed by Kuh (2009); wherein appropriately designed positive activities which enhances students engagement are the major factors contributing to desirable collegiate outcomes.

Strategies (Curriculum Planning and other feature activities) - Looking into Table 1, most institutions

emphasizes on the creation of various activities that are geared toward achieving their goals and objectives. For instance, NCU emphasizes on creating a physical environment conducive for learning with the help of the freshmen tutoring programs, accommodations and innovations in general education, and many others. While, NTHU places service learning, social research and career exploration, along with other learning activities into their curriculum programs, as well as opening a third semester (during the summer and winter vacations), as their strategies. For NCCU curriculum planning are focused more on career exploration, service learning, creativity, leadership and communication ability, along with other activities, which are all supplemented by a tutorial system and goal oriented activities. Lastly, TU utilizes systematized curriculum and activities, which are integrated into the students’ compulsory courses and activities. Programs include courses in general education, creative arts, and life education studies.

Table 1

Presence and Overall Working of Internal Quality Assurance

HEI Type Since Name Scale Core values Objectives

NTHU

Science 2008 Tsing Hua College

150 students per S.Y. since 2008 (approx. 450 students) Learn while living Interdisciplinary learning Develop a sense of curiosity Self-learning ability Strategies

1. Service learning programs

2. Social inquiry programs: Caring and participating in the community 3. Career exploration programs: Self-learning ability and aptitude development

4. Feature activities: Learning families, dormitory activities, spontaneous group, third semester (winter and summer vacation), and many others

http://www.college.nthu.edu.tw/files/11-1090-2084-1.php

NCCU

Science 2008 CCU Elite Program

80 students per S.Y. since 2008 (approx. 200 students) Holistic education Emphasis on learning by doing Expansion of viewpoint, develop international perspective Provides a model for

learning Enhances humanistic qualities Develop independent learning skills Enhance the competitiveness of the workplace Strategies

1. Curriculum design: A total of two years, one course program per semester, including career exploration, service learning, creativity, leadership, and communication ability. In addition, the student should learn two additional foreign languages

2. Workshop and seminars: Utilizing extra-curricular hours, activities includes lectures of outstanding individuals of various fields of expertise, social care activities, social services, cultural activities, technology workshops, foreign language study camps, company field trips, and many others

3. Dream schemes: Includes international volunteering, exchange students program, individual or students group project proposal, or participate in a company funded project

4. Faculty advising and guidance program: Each group is assigned with an adviser. Graduates of the CCU Elite Programs are assigned as counselors that assist in the program

(6)

Table 1 … continued

HEI Type Since Name Scale Core values Objectives

NCU

Non-Science 2008

Chengchi College Project

All freshmen students (approx. 2000 students)

Holistic education

Develop a much wider sense of thinking Develop a keener sense of

observation

Develop an independent study habit, value judgment, and the ability to reflect Empathy for others

Strategies

1. Environmental: Construction of a wooden footpath (trail), auditorium, creative laboratories, liberal arts study hall, foreign language self-learning center, mountain learning center, and dormitory food hall

2. Freshmen student counseling: Includes a freshmen orientation camp and tutoring system. Since 2009, with the integration of the university wide resources a Holistic Development

Self-Management System is implement

3. Learning dormitories: Students can select their dormitory roommate with respect to their living attitudes. Furthermore, guidance and counseling are undertaken by college advisers.

Construction of the dormitory food hall and mountain learning center.

4. Liberal arts innovation: In cooperation with the general education center, open program courses such as humanities, social sciences, general science, and many other leading general course programs. In addition, a diverse of art related activities and creative laboratory activities http://college.nccu.edu.tw/

TU

Non-Science 2008 Liberal Arts College

100 students per S.Y. since 2008 (approx. 300 students) Life education & Character education Self-exploration Social concerns Develop an understanding of other cultures and their histories

Strategies

1. Liberal Arts General Education: A Literature and art, history and current affairs, social analysis and moral reasoning, science and technology, foreign cultures and languages, etc.

2. Six art courses: Activities in the college involves the development of the six arts, namely: Rites (Rituals), Music, Archery, Riding, Writing, and Arithmetic. Contains four activities of the six arts with appreciation course programs such as music appreciation, arts, calligraphy, communication and presentation

3. Consciousness: Includes experiential learning, services learning, international relations, career planning

4. Life education: Includes lessons in life manners

5. Featured activities: Includes student outcome presentation, opening ceremonies, self-learning activities, retreat week and warm-up week, and many others

http://www3.thu.edu.tw/slal/

3. Educational outcome gains

With the various literature available for the theory of student engagement, wherein findings are said to be predictive for effective educational practices (Kuh, 2001, 2003). Student engagement actually emphasizes on the quality of student-faculty interaction (Cox & Orehovec, 2007; Hu & Ching, 2012; Hu, Hung et al., 2015), which the effects are said to be persistent and provide a value adding effect that instill future student success (Kuh, Cruce, Shoup, Kinzie, & Gonyea, 2008; Wolf-Wendel, Ward, & Kinzie, 2009; Zhao & Kuh, 2004). In light of these findings, the current study used a revised Student Engagement model specifically tailored for Taiwan students (Hu, Ching, & Chao, 2012) and adopts its various educational outcome gains, which is also based from the previous findings of Kuh and his associates (2001, 2003, 2009). These educational outcome gains are the

general education gain, practical competence gain, and personal social gain (for more information, please see Hu, Ching, & Chao, 2012). Details of the three educational outcome gains are as follows:

General education gain (GEG) - General education gain is basically consist of the items such as

having clear and effective expression in speaking and writing, which are also goals of the writing

camp of the Chengchi College Project. While the improvement of the freshmen Chinese language and

writing ability program of the TU are also perfect examples of GEG activities. In essence, the goal of the residential college model which is to uplift the general education programs of the collegiate students is undoubtedly similar to the GEG of the student engagement constructs.

Practical competence gain (PCG) - Practical skills such as critical and analytical thinking including

(7)

workplace competitiveness, and obtained work related skills have all corresponds to the objectives of the CCU Elite Programs. While the Chengchi College Project emphasis on the expanding the students’

viewpoints and having a much wider sense of thinking are all practical skills that future graduates should possess. Similarly, the Tsing Hua College emphasis on diversity of learning is all in all ingredients for having practical competence gain.

Personal social gain (PSG) - The concept of knowing thyself, developing personal values and code of

ethics, self-exploration or career exploration, understanding of cultural and ethnic differences,

developing an international outlook, social care and service, and foreign language and culture course

programs are all evidence of personal social gain and in fact inherent to the residential college model of learning.

In summary, the residential college system and the concept of student engagement by Kuh (2009) are the two leading theory for uplifting the quality of higher education. Based on such notion, the current study shall focus on the intersection of these two concepts, while looking into the effectiveness of the residential college model in Taiwan. This study also provides the much needed empirical data for strengthening the concepts of residential colleges systems of learning. Furthermore, with the residential college learning model deemed as informal learning, which the outcome of such is hard to assess (difficult to apply on the existing college evaluation system). The development of a model of learning in residential colleges shall make available a psychometrically sound tool for accessing how Taiwan college students learn.

4. Methodology

4.1 Design

Within studies in behavioral sciences, differences among experimental groups are said to be the cause of error variance. Such variations in the dependent variable can be considered as nuisance variation. Hence, in order to control this effect, the current study uses the block design as a method in reducing error variance contributed by background demographic variables (Kirk, 2009, 2013). For instance, in a case wherein participants are all of the same year level, hence, it would be practical to hold the nuisance variable year level constant. Similarly, the researcher can design a study by assigning the experimental units randomly to the treatment levels so that the known and unsuspected sources of variation among the units are distributed over the entire experiment and thus do not affect just one or a limited number of treatment levels. Lastly, the nuisance variable can also be set as one of the factors in the experiment itself (for more information please read Kirk, 2013).

Such approach in using the nuisance variable as one of the factors in the experiment is done in order to isolate the variation attributes, so that it does not appear in the estimates of the treatment of error effects. The actual procedure involves forming n blocks of p homogeneous experimental units, wherein p is the number of levels of the treatment and n blocks corresponds to the levels of the nuisance variable. These blocks are then formed so that at the start of the experiment, the experimental units in each block are more homogeneous with respect to the nuisance variable than are those in different blocks (Kirk, 2013, p. 280). With this having said, the current study shall use the factor school as a nuisance variable, since the variable whether a student is enrolled

in a residential college or not is crucial. However, there is a certain dilemma, since residential colleges are organized within unique institutions (schools/universities). In order to remedy this issue, within the current study, the researchers shall considers school as a block variable, while residential college as treatment variable.

Within the experimental design model, a score Y in a randomized block design is a composite that reflects the effects of treatment j, block i, and all other sources of variation that affects Y. These other sources of variation are collectively referred to as error effects (Kirk, 2013, p. 285). Y can therefore be expressed as:

(8)

= μ + + + ( = 1, … , ; = 1, … , ) （formula 1） Wherein,

Y is the score in the ith block and jth treatment level;

μ is the grand mean of the population means, µ11, µ12, …, µnp. The grand mean is a constant for all

scores in the experiment;

is the treatment effect for population j and is equal to .− , the deviation of the grand mean from the jth population mean. The jth treatment effect is a constant for all scores in treatment level and is subject to the restriction ∑ = 0;

is the block effect for population i and is equal .− , the deviation of the grand mean from the

ith population mean. The block effect is a random variable that is NID (0, );

is the error effect associated with Y and is equal to − .− .+ . The error effect is a random variable that is NID (0, _!) and independent of .

School is therefore considered as the block. Whereas Y_"# is the student learning outcome. While, μ is the

mean student learning outcome. α_# is the residential college and π_" is the school effect. ε_"# as the error. Y_"# possess no interaction between residential college and school.

4.2 Covariance

The values of the parameters μ, , _" and _"# are unknown, but they can be estimated from sample data as follows:

Parameters of the model equation

= μ + + + Sample estimators of the parameters

= Y*.. + +Y*.− Y*.., + (Y*.− Y*..) + (− Y*.− Y*.+ Y*..) Score Grand mean Treatment effect Block effect Residual effect

Within the experimental design model, a score Y in a randomized block design is a composite that reflects the effects of treatment j, block i, and all other sources of variation that affect Y. These other sources of variation are collectively referred to as error effects (Kirk, 2013, p. 285). Y can therefore be expressed as:

= μ + + + ( = 1, … , ; = 1, … , ) （formula 2） Y is the score in the ith block and jth treatment level;

(9)

μ is the grand mean of the population means, µ11, µ12, …, µnp. The grand mean is a constant for all

scores in the experiment;

is the treatment effect for population j and is equal to .− , the deviation of the grand mean from the jth population mean. The jth treatment effect is a constant for all scores in treatment level and is subject to the restriction ∑ = 0;

is the block effect for population i and is equal .− , the deviation of the grand mean from the

ith population mean. The block effect is a random variable that is NID(0, );

is the error effect associated with Y and is equal to − .− .+ . The error effect is a random variable that is NID (0, _!) and independent of .

Figure 1. CR-3 and RB3 covariance distribution

For example, partition of the total sum of squares and degrees of freedom for a CR-3 design with 30 students in 3 groups. The estimate of the experimental error for the RB-3 design will be less than the estimate for the CR-3 design if SSBLOCKS in the RB-3 design accounts for an appreciable portion of the total sum of squares because SSRESIDUAL = SSWG – SSBLOCKS.

For the randomized block design is greater than the F statistics for the completely randomized design:

F = ../0/2 ..30/27= 5./0 5.30 F = ..6/2 ..78.9:;6</18= 5.6 5.78.9:;6<

SSRESIDUAL is equal or just slightly less than SSWG because the latter design has more degrees of freedom to estimate error variance and requires a smaller critical value to reject the null hypothesis.

(10)

The values of the parameters μ, , _" and _"#are unknown, but they can be estimated from sample data as follows:

Parameters of the model equation

= μ + + + Sample estimators of the parameters

= Y*.. + +Y*.− Y*.., + (Y*.− Y*..) + (− Y*.− Y*.+ Y*..) Score Grand mean Treatment effect Block effect Treatment error

4.3 Research tools and process

To collect the data, during the end of the school year 2014-15, an online survey is setup and distributed through the various residential colleges, at the same time a call for participation was also made on the university student affairs offices encouraging volunteer student participants. The survey content are based on the previous findings of Hu, Ching, and Chao (2012) regarding the Taiwan model of student engagement. In addition, other pertinent issues such as civility, time spent on activities, and various educational objectives were also included within the survey. The survey lasted for three weeks with a total of 1359 respondents; 536 residential college and 823 non-residential college students.

The study uses two independent variables; one is the school which is also the control variable and the

nuisance factor, while the other is whether the student is enrolled in a residential college (RS) or regular/non-residential college (NS). For this study the dependent variable was used to test the effectiveness of learning outcomes in terms of general education gain (GEG), practical competence gain (PCG), and personal social gains (PSG). GEG is basically consists of the items such as having clear and effective expression in

speaking and writing. PCG include the practical skills such as critical and analytical thinking including the

development of good study habits, value judgment, ability to give reflections, enhancing the workplace

competitiveness, and obtained work related skills. While, PSG is the concept of knowing thyself, developing

personal values and code of ethics, self-exploration or career exploration, understanding of cultural and ethnic

differences, developing an international outlook, social care and service, and foreign language and culture

course programs (for more information on the survey items, please see Hu, Ching, & Chao, 2012).

5. Results

5.1 Differences in civility

The succeeding analysis is computed from 536 residential college (RS) and 823 non-residential college students (NS) participants of the study. Based from previous literature, it is noted that a key factor for successful student-faculty interaction is the friendship atmosphere among both the students and faculty (Connelly, 2009; Marchiondo & Marchiondo, 2010); hence, civility within the institutions is a vital factor of residential colleges. Table 2 shows the t-test results of the various RS and NS in the different institutions. Table 2 shows the various results noting the obvious differences between RS and NS civility within NTHU, NCU, and TU, while students in NCCU having varied perceptions except towards school staff. These findings clearly shows that within residential colleges, students tend to spend more time together, while at the same time having increased opportunity to interact with their teachers and administrative staff. Furthermore, previous studies have noted that interactions really do matter in terms of students’ future (Kuh, Kinzie, Schuh, & Whitt, 2005), more specifically creating the notion that teachers provide opportunities for all students (Hurtado et al., 2011; Lundberg & Schreiner, 2004). Lastly, findings also echoes the results of a previous study noting the importance of quality

(11)

student and faculty communications and interactions, which is found within the residential model of learning (Hu, Hung et al., 2015).

Table 2

Differences in perceived civility among RS and NS students (N=1359)

Items NTHU NCCU NCU TU

RS NS t RS NS t RS NS t RS NS t Students Friendship 5.89 5.38 4.90*** 5.78 5.52 1.66 5.88 5.29 4.13*** 5.45 5.13 2.00* Support 5.41 4.98 3.92*** 5.46 5.10 2.18* 5.38 4.81 3.74*** 5.09 4.77 2.05* Belongingness 5.35 4.78 4.66*** 4.95 4.76 0.93 5.26 4.39 5.11*** 5.09 4.69 2.31* Faculty Approachable 5.67 4.70 7.75*** 5.27 5.10 0.88 5.99 4.62 8.98*** 5.55 5.13 2.58* Helpful 5.54 4.80 5.90*** 5.33 5.11 1.20 5.94 4.67 8.19*** 5.56 5.13 2.69** Empathy 5.68 4.68 8.86*** 5.27 4.99 1.56 5.78 4.84 6.49*** 5.60 5.04 3.62*** School staff Helpful 4.81 4.36 3.48** 5.55 4.48 5.83*** 5.88 5.29 4.13*** 5.54 4.46 6.24*** Caring 4.94 4.25 5.28*** 5.28 4.38 4.90*** 5.38 4.81 3.74*** 5.44 4.17 7.13*** Flexible 4.64 3.90 5.55*** 5.24 4.02 6.28*** 5.26 4.39 5.11*** 5.01 3.99 5.29*** Note. * p < .05. ** p < .01. *** p < .001. Items are in 7-point Likert scale.

NTHU: RS=202, NS=308; NCCU: RS=93, NS=168; NCU: RS=129, NS=153; TU: RS=110, NS=216.

5.2 Differences in time spent on activities

As for the difference within time spent on activities, Table 3 shows the various results noting variation in time spent on the different activities. On initial conception of the items, it is hypothesized that RS would tend to spend more time on extra-curricular activities such as: joining clubs and sports activities. In addition, RS would have more opportunity to consider a part-time job within the school, such as teaching and/or research assistant. An added note is that since RS are staying in the university dorm (residential houses) less time is spent on transportation (commuting) to and from classes, however, finding seems to vary from school to school. These findings suggest that although there are significant differences between the time spent by RS and NS, however, the differences are still quite contextual and not consistent across different universities.

Table 3

Differences in time spent on activities among RS and NS students (N=1359)

RS NS t RS NS t RS NS t RS NS t Homework 3.67 3.16 3.75*** 3.06 3.14 -0.48 3.26 2.72 3.69*** 3.31 3.33 -0.13 In-school job 1.63 1.75 -1.07 1.96 1.63 2.04* 1.82 1.58 1.68 1.70 1.94 -1.41 Out-school job 1.32 1.50 -1.82 1.49 1.36 1.00 1.88 1.47 2.70** 1.45 1.88 -2.17* Extra-curricular 3.26 2.58 4.99*** 2.78 2.53 1.28 3.01 2.59 2.17* 2.53 2.62 -0.45 Social activities 3.55 3.18 2.99** 3.35 3.30 0.27 3.53 3.43 0.57 3.74 3.58 0.83 Internet (academic) 3.06 2.91 1.38 3.15 2.99 0.89 3.22 2.84 2.43* 3.31 3.10 1.18 Internet (leisure) 4.05 3.64 2.78** 3.25 3.96 -3.49** 3.71 3.42 1.73 3.65 3.94 -1.51 Transportation 1.57 1.79 -2.69** 1.78 1.74 0.49 1.86 1.88 -0.12 1.82 2.01 -1.53 Exam preparation 1.79 2.41 -5.10*** 2.67 2.03 3.03** 2.11 2.16 -0.31 2.19 2.61 -2.38* Note. * p < .05. ** p < .01. *** p < .001. Items are in 5-point Likert scale.

5.3 Differences in educational objectives

As noted with the contextual nature of residential colleges, Table 4 shows the comparison of the different educational objectives of the four institutions. Results show that four distinct objectives have significant differences between RS and NS. Findings suggest that for the objective “Have students spend more time studying” with NS scoring higher mean scores ranging from 3.99 to 4.64, actually holds true since residential colleges promote more extra and co-curricular activities. Results also show that these three objectives “Encourage

(12)

and “Provides students with appropriate social networking”, RS scored significantly higher than NS; denoting that residential colleges promotes interdisciplinary knowledge, encourage nationalism, and interaction among peers.

Table 4

Differences in perceived educational objectives among RS and NS students (N=1359)

RS NS t RS NS t RS NS t RS NS t

Have students spend more time studying 2.33 4.64 -16.07*** 3.52 4.12 -3.09** 3.22 3.99 -3.83*** 3.40 4.27 -4.88*** Provide students with adequate educational resources 3.60 5.14 -11.11*** 4.26 4.96 -3.76*** 4.66 4.66 -0.01 4.47 4.70 -1.27 Encourage interaction between students of different major 6.01 4.73 10.06*** 5.87 4.65 6.15*** 6.02 5.16 5.78*** 5.85 4.98 5.05*** Develop students that will become future responsible citizens 4.82 3.92 6.13*** 5.01 4.20 4.35*** 4.92 4.08 4.41*** 5.55 4.52 5.61*** Provides students with appropriate social networking 5.51 4.41 8.75*** 5.63 4.61 5.68*** 5.26 4.68 3.61*** 5.69 4.85 4.80*** Encourage students to participate in co- and extra-curricular activities 5.38 4.39 7.45*** 5.53 4.82 3.86*** 4.91 4.71 1.12 5.62 4.86 4.49*** Strengthen learning through the use of information technology 4.12 5.12 -7.65*** 4.90 5.30 -2.19* 4.53 4.54 -0.01 4.75 4.88 -0.74

Note. * p < .05. ** p < .01. *** p < .001. Items are in 7-point Likert scale.

5.4 Differences in educational outcome gains

Statistical analysis shows that for GEG RS mean scores ranges from 4.56～4.86 with Standard Deviation (SD) ranging from 1.01～1.19, while NS have a mean score of 4.42～4.76 (SD 0.96～1.24). PCG RS mean score ranges from 3.99～4.42 (SD 0.95～1.04), while NS mean score ranges from 4.55～5.04 (SD 0.95～1.24). Lastly, PSG RS mean score ranges from 5.06～5.30 (SD 0.99～1.20), while NS mean score ranges from 4.71～ 4.91 (SD 0.96～1.21). For more details of the scores, please see Table 5.

As for the block analysis, results show that GEG, student role (whether RS or NS) results with F (1, 1354) = 1.69 (p>.05), denotes that there are no significant difference between the two groups of students. With regards to PCG, student role results with F (1, 1354) = 141.35 (p<.001), denotes that there exists a significant difference between the two groups (NS scoring higher than the RS). Lastly, PSG results with F (1, 1354) = 31.99 (p<.001), denotes that there exists a significant difference for the two groups of students, suggesting that RS scored higher than the NS. For more information please see the Tables 6 to 8.

(13)

Hu, Y.-L., Hung, C.-H., Ching, G. S., & Hung, G.-C. Table 5

Demographic background of participants (N=1359)

School Factor Type n M SD

NTHU GEG RS 202 4.56 1.07 NS 307 4.51 0.96 PCG RS 202 3.99 1.04 NS 307 5.04 1.03 PSG RS 202 5.17 1.08 NS 307 4.84 0.96 NCCU GEG RS 93 4.78 1.05 NS 168 4.69 0.96 PCG RS 93 4.27 1.00 NS 168 5.09 0.95 PSG RS 93 5.23 1.14 NS 168 4.92 0.97 NCU GEG RS 131 4.63 1.01 NS 132 4.42 1.24 PCG RS 131 4.42 0.95 NS 132 4.55 1.24 PSG RS 131 5.06 0.99 NS 132 4.71 1.21 TU GEG RS 110 4.86 1.19 NS 216 4.76 1.15 PCG RS 110 4.32 1.01 NS 216 4.81 1.15 PSG RS 110 5.30 1.20 NS 216 4.87 1.19

Note. NS – regular or non-residential college students, RS – residential college students. For the NS: NCU students encompass 12% of the total participants.

Table 6

General education gains analysis

Source df SS MS F p

Role 1 1.92 1.92 1.69ns .194

School (block) 3 18.62 6.21 5.44** .001

Residual 1354 1544.29 1.14

Total 1358 1564.84

Note. ns = not significant, p > .05. ** p < .01. Table 7

Practical competence gain analysis

Source df SS MS F p Role 1 160.07 160.07 141.35*** .000 School (block) 3 13.34 4.45 3.93** .008 Residual 1354 1533.28 1.13 Total 1358 1706.68 Note. ** p < .01. *** p < .001. Table 8

Personal social gains analysis

Source df SS MS F p

Role 1 37.39 37.39 31.99*** .000

School (block) 3 3.45 1.15 0.98ns .400

Residual 1354 1582.65 1.17

Total 1358 1623.48

(14)

6. Conclusions

The current study attempts to explain the various implications of studying in a residential college system in terms of the educational outcome gains within the theory of student engagement. Furthermore, the study also employed the use of the statistical block analysis to provide further explanation of the effects of school (RS or NS). Results show that there exist significant differences between the RS and NS students’ practical and social competencies, while no difference were found on the students’ general competencies. These results could be attributed to that students within a residential model are more focused on beyond academics; more specifically increased opportunity for interactions either between peers and/or faculty, hence, increase social competency gains. However, with regards to practical gains, NS seems to gain the upper hand as compared to their RS counterparts. Further in-depth analysis is needed to clarify this result. Lastly, it is also not surprising that RS and NS show no significant differences with respect to their general competency gains, since both types of students goes to similar classes with similar teachers, hence, undergoes similar in-class education. In essence, the randomized block analysis results show that residential colleges are unique in nature, focusing not only domain specific knowledge, but more on diverse skills including nationalism and active participation. Findings should be able to help higher education institutions design better course programs and activities needed in further enhancing the faculty-student interaction towards more student engagement.

Acknowledgement: This work is supported in part by the Taiwan Ministry of Science and Technology projects 101-2410-H-004-168-MY2 and 104-2410-H-004-151-SS2.

7. References

Astin, A. W. (1977). Four critical years: Effects of college on beliefs, attitudes, and knowledge. San Francisco: Jossey-Bass.

Chen, Y. L. (2014). Teaching students the traits that matter in Taiwan: A structured conceptualization approach.

International Journal of Research Studies in Psychology, 3(4), 29-47. http://dx.doi.org/10.5861/ijrsp.2014.827

Chen, Y. L., & Ching, G. S. (2012). Emerging global literacy: What college graduates ought to be. In 2012

Conference on Creative Education (pp. 730-733). Irvine, CA: Scientific Research Publishing. Ching, G. S., Lien, W.-C., & Chao, P.-C. (2014). Developing a scale to measure the situational changes in

short-term study abroad programs. International Journal of Research Studies in Education, 3(5), 53-71. http://dx.doi.org/10.5861/ijrse.2014.771

Chou, C. P., & Ching, G. S. (2015). Cross-straitization of higher education: Voices of the mainland Chinese students studying in Taiwan. International Journal of Information and Education Technology, 5(89-94). http://dx.doi.org/10.7763/IJIET.2015.V5.482

Connelly, R. J. (2009). Introducing a culture of civility in first-year college classes. The Journal of General

Education, 58(1), 47-64. http://dx.doi.org/10.1353/jge.0.0029

Cox, B. E., & Orehovec, E. (2007). Faculty-student interaction outside the classroom: A typology from a residential college. The Review of Higher Education, 30(4), 343-362.

http://dx.doi.org/10.1353/rhe.2007.0033

Duke, A. (1996). Importing Oxbridge: English residential colleges and American universities. New Haven: Yale University Press.

Durrani, M., & Khan, S. (2009). Bringing the residential college model to LUMS. Retrieved from http://web.lums.edu.pk/~sohaib/docs/ResidentialCollegeWhitePaper.Short.pdf

Gary, R. P., Charles, S. S., & Thomas, R. B. (1997). Enhancing the educational impact of residence halls.

Journal of College Student Development, 38(6), 609-622.

Harvard University. (2010). Final report of the task force on general education: Harvard university faculty of art and science. Retrieved from http://www.sp07.umd.edu/HarvardGeneralEducationReport.pdf

Hu, Y.-L., & Ching, G. S. (2012). Factors affecting student engagement: An analysis on how and why students learn. In 2012 Conference on Creative Education (pp. 989-992). Irvine, CA: Scientific Research Publishing.

(15)

52 Consortia Academia Publishing

overview of psychometric properties. International Journal of Research Studies in Education, 1(1), 69-90. http://dx.doi.org/10.5861/ijrse.2012.v1i1.19

Hu, Y.-L., Ching, G. S., & Hung, C.-H. (2015). Comparison of concepts within the residential colleges in Taiwan.

International Journal of Information and Education Technology, 5(12), 936-940. http://dx.doi.org/10.7763/IJIET.2015.V5.641

Hu, Y.-L., Hung, C.-H., & Ching, G. S. (2015). Student-faculty interaction: Mediating between student engagement factors and educational outcome gains. International Journal of Research Studies in

Education, 4(1), 43-53. http://dx.doi.org/10.5861/ijrse.2014.800

Hurtado, S., Eagan, M. K., Tran, M. C., Newman, C. B., Chang, M. J., & Velasco, P. (2011). "We do science here": Underrepresented students' interactions with faculty in different college contexts. Journal of

Social Issues, 67(3), 553-579. http://dx.doi.org/10.1111/j.1540-4560.2011.01714.x

Kirk, R. E. (2009). Experimental design. In R. E. Millsap & A. Maydeu-Olivares (Eds.), The SAGE handbook of

quantitative methods in psychology (pp. 23-45). Thousand Oaks, CA: Sage.

Kirk, R. E. (2013). Experimental design: Procedures for behavioral sciences (4th ed.). Thousand Oaks, CA: Sage.

Kuh, G. D. (2001). Assessing what really matters to student learning inside the national survey of student engagement. Change: The Magazine of Higher Learning, 33(1), 10-17.

http://dx.doi.org/10.1080/00091380109601795

Kuh, G. D. (2003). What we're learning about student engagement from NSSE: Benchmarks for effective educational practices. Change: The Magazine of Higher Learning, 35(2), 24-32.

http://dx.doi.org/10.1080/00091380309604090

Kuh, G. D. (2009). The national survey of student engagement: Conceptual and empirical foundations. New

Directions for Institutional Research, 141, 5-21. http://dx.doi.org/10.1002/ir.283

Kuh, G. D., Cruce, T. M., Shoup, R., Kinzie, J., & Gonyea, R. M. (2008). Unmasking the effects of student engagement on first-year college grades and persistence. The Journal of Higher Education, 79(5), 540-563. http://dx.doi.org/10.1353/jhe.0.0019

Kuh, G. D., Kinzie, J., Schuh, J. H., & Whitt, E. J. (2005). Student success in college: Creating conditions that

matter. Washington, DC: Jossey-Bass.

Lundberg, C. A., & Schreiner, L. A. (2004). Quality and frequency of faculty-student interaction as predictors of learning: An analysis by student race/ethnicity. Journal of College Student Development, 45(5), 549-565. http://dx.doi.org/10.1353/csd.2004.0061

Marchiondo, K., & Marchiondo, L. A. (2010). Faculty incivility: Effects on program satisfaction of BSN students. Journal of Nursing Education, 49(11), 608-614.

http://dx.doi.org/10.3928/01484834-20100524-05

National Study of Living-Learning Programs. (2007). Purpose and rationale. Retrieved from http://www.livelearnstudy.net/studydesign/purposeandrationale.html

Pike, G. R. (1999). The effects of residential learning communities and traditional residential living arrangements on educational gains during the first year of college. Journal of College Student

Development, 40(3), 269-284.

Ryan, M. B. (1992). Residential colleges: A legacy of living and learning together. Change: The Magazine of

Higher Learning, 24(5), 26-35. http://dx.doi.org/10.1080/00091383.1992.9937126

Shapiro, N., & Levine, J. (1999). Creating learning communities: A practical guide to winning support,

organizing for change, and implementing programs. San Francisco: Jossey-Bass.

Smith, T. B. (1994). Integrating living and learning through residential colleges. In C. C. Schroeder & P. Mable (Eds.), In realizing the educational potential of residence halls (pp. 241-266). San Francisco:

Jossey-Bass.

Wawrzynski, M. R., & Jessup-Anger, J. E. (2010). From expectations to experiences: Using a structural typology to understand first-year student outcomes in academically based living-learning communities. Journal

of College Student Development, 51(2), 201-217. http://dx.doi.org/10.1353/csd.0.0119

Wawrzynski, M. R., Jessup-Anger, J. E., Stolz, K., Helman, C., & Beaulieu, J. (2009). Exploring students' perceptions of academically based living-learning communities. College Student Affairs Journal, 28(1), 138-158.

Wolf-Wendel, L., Ward, K., & Kinzie, J. (2009). A tangled web of terms: The overlap and unique contribution of involvement, engagement, and integration to understanding college Student Success. Journal of College

Student Development, 50(4), 407-428. http://dx.doi.org/10.1353/csd.0.0077

Zhao, C.-M., & Kuh, G. D. (2004). Adding value: Learning communities and student engagement. Research in