Assessment Criteria in the Context of English-taught Courses
Min-Chun Ku
Assistant Professor, Graduate Institute of Information and Library Science Tamkang University, Taiwan (R.O.C.)
E-mail: [email protected]
Keywords: Document Assessments; Genre Theory; Information Use; Work Tasks
【Abstract】
This study investigated the associations among the document genres faculty used to support their teaching, the criteria they employed to assess these genres and the tasks they performed to use these genres in English‐taught courses. It first employed qualitative citation analysis to identify the genres faculty used based on their teaching materials. It then employed semi‐structured interviews to collect data regarding how they assessed and used these genres. Interview transcripts were analyzed using qualitative content analysis and co‐occurrence analysis. A total number of 17 faculty members from different disciplines participated, and together they contributed 19 undergraduate and master courses.
Results demonstrate different genres have different information attributes, and the assessment criteria faculty employed reflected such attributes. These attributes allowed faculty to use genres to perform their tasks.
【Long Abstract】
Introduction
This study extended Ku’s (2015) research on the associations among document genres, information use tasks, and credibility assessments in the context of university teaching in the U.S.A. It included all the assessment criteria faculty employed and adopted the same methods for subsequent theoretical comparison. It recruited faculty who taught English-taught courses at Tamkang University in Taiwan as samples and explored the associations among genres, tasks and criteria.
DOI: 10.6245/JLIS.201810_44(2).0003
Definitions of Key Concepts
Genre is a distinctive type of communicative action emerging from recurrent situations in which social interactions among community members become typified over time. The genre of a document is often characterized by and could be identified based on its socially recognized communicative purposes and forms (Crowston & Kwaśnik, 2003; Crowston, 2010). Some genres are defined primarily based on their communicative purposes, such as call-for-participation and news; some are defined primarily based on their forms, such as tables and rating scale; still some are defined based on a fusion of communicative purposes, forms and content, such as dictionary and journal articles (Crowston, 2010). Genre repertoire is defined as the variety of documents faculty assess and use to support their teaching, including both printed and digital documents.
Tasks were classified into teaching tasks and information use tasks. Teaching a course is a purposeful, goal-oriented activity that motivates faculty to engage in different types of information activities. In the context of information seeking and use, tasks can be classified into work tasks and information tasks.
The former refers to the activities people perform to fulfill the requirements of their work responsibilities.
Work tasks generate problems and further information needs, which trigger information seeking and use because solving problems requires information (Byström & Hansen, 2005; Vakkari, 2003). Information use task refers to the activities faculty engage in to utilize the information within the documents they have obtained (Freund, 2008; Kari, 2010). It is guided by faculty’s teaching goals.
Document assessment criteria refer to faculty’s perceived values or utility of documents. Barry and Schamber (1998) compared the two studies they conducted and found users employ a finite range of criteria when judging how relevant a document might be to their information needs across different information seeking contexts. These criteria include: depth/scope/specificity, accuracy/validity, clarity, currency, tangibility, quality of sources, accessibility, availability of information/sources of information, verification, and affectiveness.
Research Methods
The author employed qualitative citation analysis and semi-structured interviews to collect data. First, citations in participants’ teaching materials (e.g., syllabi and lecture slides) were analyzed to identify the documents they used and genres of these documents. These were documented in Excel files called customized genre repertoires. The facet of genre was highlighted in yellow to help participants understand genres intuitively. Each participant received a customized genre repertoire before interviews.
Next, interviews were conducted to elicit participants’ assessments and their tasks. Interviews were fully
transcribed. Ku’s (2015) three coding schemes on genres, teaching and information use tasks, and evaluative and assessment criteria, were adopted to analyze transcripts. Open coding was also employed to identified participants’ tasks and criteria inductively (Neuendorf, 2001; Schreier, 2012), which were then integrated into Ku’s (2015) coding scheme. The author then distinguished the tasks participants performed to use a specific document or genre and the criteria they employed to assess a specific document or genre. Furthermore, co-occurrence analysis was conducted to identify the tasks associated with specific criteria when participants used a specific document or genre. Finally, the documents identified as the same genres were aggregated to identify how often a task was associated with a specific criterion.
The author obtained the complete list of English-taught courses opened in the academic year of 2014.
The faculty on this list were recruited. The author also visited the websites of English-taught programs to recruit other faculty. Both face-to-face and telephone interviews were conducted.
Results The Courses
A total number of 17 faculty contributed 19 courses to this study. Among all, fourteen courses were opened for first to third year undergraduate students. Five were opened at the master level. Thirteen were required, while six were elective. One was developed based on the problem-based learning principles and the participant won a teaching award for it. Some participants taught two classes for a course, while some taught one class. The departments in which participants were employed included: information engineering, international business, transportation management, information and library science, math, accounting, and so on. Several participants taught students in other schools because their courses were required by other schools or classified as general education, such as calculus for all the departments in the School of Business and Management. Almost all the participants were full-time faculty. They had taught their courses from one time to more than ten times.
The Associations Among Genres, Tasks, and Assessment Criteria
The co-occurrence analysis reveals the following results: (1) “Providing foundational text” was often performed with textbooks and maps. It was often associated with “recency” and “organization.” (2) Diverse genres (e.g., games, maps and annual reports) were used to perform “enable students to understand an area/topic”, which was often associated with “organization”, “completeness” and
“endorsed usage.” (3) “Complementing/supplementing other resources” was often performed with research genres. It was often associated with “recency”, “topic match” and “depth.” (4) Diverse genres
(e.g., rating scales and job application forms) were used to “provide an example(s)”, which was associated with multiple criteria. (5) Instructional videos and visual genres were often used to “engage students”, which was often associated with “friendliness.” (6) Visual genres and instructional videos were often used to “improve students’ understanding”, which was often associated with “friendliness.”
(7) Diverse genres (e.g., reports, textbooks and Tedtalks) were used to “present reality”, which was often associated with “source”, “recency” and “completeness.” (8) Diverse genres (e.g., annual reports and online encyclopedia entry) were often used to “connect with the real world/make a connection(s)”, which was often associated with “topic match.” (9) Diverse genres were used to “provide suggested readings/more information”, which was often associated with “depth” and “recency.” (10) Some tasks, including “follow the university’s rules”, “have a quiz” and “to reinforce”, were performed rarely. Thus, it was challenging to identify their associations with specific criteria.
Conclusions
Different genres have different information characteristics, which were reflected in the criteria faculty employed. Faculty utilized these characteristics to perform different information use tasks. Some of the genres, tasks, and criteria were specific to the English-taught context. Faculty assessed and used genres according to students’ nationalities, helped local students understand the content, connected international students and enhanced their engagement. Some commonly used genres, including textbooks and instructional videos, were used to improve students’ English and develop professional vocabulary.
【Romanization of references is offered in the paper.】