CELT Newsletter, Spring 2007

Table of contents

Shifting the Program Design Focus: In Preparation for the 4-Year Degree ... .1-9

CLI Fifth Round Awarded Projects have begun! ...... 10-11 Tech Camp 2006 Winter.. ... .12 Recent Events Highlights ... 13

Spring 2007

Shifting thB Program DBsign Focus:

In

PrBparation

for

thB 4-YBar DBgrBB

HKUST

is on its way to the implementation of a four-year degree. One major implication of this is that some thought is required in planning for the curriculum, courses, and assessments. In this edition, we will look at "Intended Learning Outcomes", including what they are in general terms and in a HKUST context, how assessments can be aligned with intended learning outcomes for the benefit of student learning and development. We will st311 by going through the types of intended learning outcomes and the desired levels to be achieved by students over a 4-year program. This will be followed by four program document examples from different universities.

Intended Learning Outcomes

_{Now let us take a closer look at the "Disciplinary} Knowledge" palt by using Bloom's Taxonomy of Cognitive Outcomes and see how the progression can go within a four-year time frame.

The Accreditation Board for Engineeting and Technology (ABET) defines the tenn "outcomes" as follows:

"Program outcomes are statements that describe what students are expected to know and be able to do by the time of graduation. These relate to the skills, knowledge, and behaviors that students acquire in their matriculation through the program".

These are the three Types of Intended Levels Learning Outcomes

types of intended

learning outcomes and Disciplinary Knowledge

Disciplinary Skills Create

the levels expected to Evaluate be achieved by students

(e.g. factual, conceplual, _Analyse procedural knowledge) Apply

over a program. _Remember Understand

Generic Skills

(e.g. team working,

Mastery communication) _Production Conformation Simulation Perception Attitudes

(e.g. ethical behavior,

Behaving

social responsibility, _Believing

professionalism) Valuing Responding

Receiving

Using Anderson et aI's Taxonomy of Cognitive Outcomes as a Framework for the Disciplinary Know/edge of 4-Year Program Design

In Anderson et ai's Taxonomy of Cognitive Outcomes, different levels of learning are identified. They move from the "simplest to the most advanced". The levels and the related cognitive processes are listed in the table below:

Levels of Learning: Cognitive Process:

Level 6: Creating Put elements together to form a coherent or functional whole; reorganize elements into a new pattern or structure

Level 5: Evaluating Make judgments based on criteria and

standards

Level 4: Analysing Break material into its constituent parts and determine how the parts relate to one another

and to an overall structure or purpose Level 3: Applying Carry out or use a procedure in a given situation Level 2: Understanding Construct meaning from instructional

messages, including oral, written, and graphic

communication

Levell: Remembering Retrieve relevant knowledge from long-term memory

Over the 4-year degree program there should be a broad development trend fi·om lower to higher cognitive outcomes which would look like this:

High-level Creating

Creating

cognitive Evaluating Evaluating

outcomes

Analysing Analysing

Low-level Applying

"'"

Applying

cognitive Understanding Understanding

outcomes

Remembering Remembering

Year 1

Year 2

Year 3

Year 4

In the early years, lower level cognitive outcomes, i.e. "Remembering" and "Understanding" are given stronger emphasis. The level moves upwards as the year moves on. Higher level outcomes like "Evaluating" and "Creating" would have more emphasis in later years. It is a curriculum team's responsibility to ensure this development over the program and to make sure the four types of intended learning outcomes are covered.

Levels of Learning: Associated Action Verbs:

In the process of writing leaming outcomes, the curriculum team would use associated action verbs for different levels of learning. Some verbs are listed in the table for consideration.

Level 6: Creating Hypothesise, Design, Construct, Plan, Invent, Devise

Level 5: Evaluating Judge, Test, Monitor, Detect, Co-ordinate

Level 4: Analysing Differentiate, Discriminate, Distinguish, Deconstruct

Level 3: Applying Use, Execute, Implement

Level 2: Understanding Clarify, Illustrate, Categorise, Predict, Compare, Contrast Level 1: Remembering Recognise, Identify, Define, Retrieve, Recall, Record <Verbs appropriate to different levels of Anderson et aI's Taxonomy>

ABC LIVE at

HKUST

-A

cademic Excellence At HKUST, program outcomes need to be

aligned with Ill(UST's seven graduate attributes - called ABC LIVE:

B

road-based education

C ompetencies and capacity building

L eadership and teamwork

I

nternational outlook

V ision and an orientation to the future

E thical standards and compassion

Shifting th8 Program D8sign Focus:

In Pr8paration for th8

4-Y8ar D8gr88

Examples of Program Documents

In this part, four assessment documents from different universities for Science,

Engineering, Business, and Social Science and Humanities will be discussed. You

will see the extracted "Learning outcomes" and "Assessment" in each example. Annotations include the type of learning outcomes and how the assessments align with those learning outcomes.

Example for Science

Name of Institute: University of Minnesota

Department: Biology

U RL: http://www.mrs.umn.edu/committees/asl/biology.shtml

The Biology Department of University of Minnesota uses summative, formative and sustainable assessments to measure students' attainment of learning in different areas. There is a very strong alignment between the intended learning outcomes and the assessments.

I { Disciplinary Knowledge )

Students Learning Outcomes: I I {

J

/ . Disciplinary Skills .

.

/ '

:==========================:

1. Students will be able to summarize the major ideas of biology / / / / ,( Disciplinary Skills and Attitudes )

accurately and concisely and to apply these ideas to a variety / , . .

of organisms and situations:.. _ .:. ______________ / / III

i

Generic Skills ) ,~---~

2. Students will be able to perform successfully a variety of "

,

r---,.

field studies and laboratory experiments:.. _________ " , This summative assessment aligns with

,

Learning Outcome 1.

3. Students will be able to prepare and present clear, cogent, , , ~===================~ and appropriate talks and papers dealing with a number of " /

biological topics:... _______________________ , /

This summative assessment aligns with Learning Outcome 1.

,

'

,

'

" This formative assessment aligns with

/ / ,, --=L=ea=r=n=in=g=o=u=tc=o=m=e=s=1=, =2=a=n=d =3=.

==:::::

Methods of Assessment:

' '

/

,-1. Unit or course examinations, term papers _______ ./ ,' / This formative assessment aligns with

I '

2. Standardized examinations ________________ , / I / ~L=ea=r=n=in=g=o=u=tc=o=m=e=s=2=a=n=d=3=.

===:::::

3. Oral presentations _____________________ ,' / / /

4. Written lab report and/or poster sessions _________ / / 5. Senior Seminar ________________________ " 6. Faculty-supervised research project and other

research-related activities _____________________ _

This sustainable assessment task aligns with Learning Outcomes 1, 2 and 3.

This formative assessment aligns with Learning Outcome 2.

7. Portfolio includes lab report and related materials, along with This sustainable assessment task aligns summaries of any additional research experiences _____ ~ " " ... with Learning Outcomes 1, 2 and 3.

' )

Shifting of Program D8sign Focus:

In Pr8paration for th8

4-Y8ar D8gr88

Example for Engineering

l

Name of Institute: The University of Texas Pan American College: Science and Engineering

Degree Program: B.S. in Mechanical Engineering

U RL: http://www.ie . utpa. edu/SL Outcomes/Spring06/CoSE/BSMechanicaIEnggSpring06. pdf

This comes from College of Science and Engineering Student Learning Outcomes Plans and Assessment Reports. The Learning Outcomes are in two parts:

1. School Level: Student Learning Outcomes

2. Department Level: Program Specific Outcomes for Mechanical Engineering

In some programs, the alignment between Learning Outcomes and Assessment might not be as clearly mapped as in this one.

School Leve

l

Students Learning Outcomes (for Science and Engineering College) (p.1) _{{ Disciplinary Knowledge )}

It will be demonstrated that the student: "

(

~=======::

)

" , Disciplinary Skills .

1. is able to use knowledge of mathematics, basic sciences and engineering " ( )

(' ,', Generic Skills

to analyze (identify, formulate, and solve) problems in mechanical ... _ _ _ _ _ _ _ ---'

.

.

'

"

englneenng. _________________________________ , " , , - - - . . ,

, Alternative verbs can be

3. is able to design mechanical devices, systems or processes that meet " " , used here to make it

given specification. _____________________________ ,',' / more specific, e.g. "able

" to explain".

5. is able to communicate ideas effectively in graphical, oral and in written _I I _, I

. media._ - - - -I / ~ Attitudes )

6.

[understand fth-e -

p

~~

f;

s

~

i

~;

a

l ;

e

~~

o-"

~

i

b

i

ii

t

~ ~f

;n

-

e

-

n

~

i

~;

e~ ~;

d

-

h

o

~

-

I I

'---~

engineering solutions impact safety, economics, ethics, politics, societal, I

cultural and contemporary issues. ____________________ _ I I

Means of Assessment that Aligns with the above Student Learning Outcomes (p.1)

1. Home work; Tests; Pre-test Examination at the end of senior year; Senior "

design project; Performance on the Fundamentals of Engineering

_,

,

Examination (FE examk _________________________ ,

I

I

These summative assessments align with Learning Outcome 1.

,

~---~

These summative

assessments align with

" Learning Outcome 3.

These summative

, assessments align with

3. Senior design Project; Building a model; Oral Presentation and written I , I Learning Outcome 5.

report. ____________________________________ ,' , I ' - _ _ _ _ _ _ _ ..J

,

5. Written and oral Presentation of senior design project __________ ,'

6. Senior design project; Engineering Ethics _____ _____ _______ -' -'

These summative assessments align with Learning Outcome 6.

Department Level

Program Specific Outcomes for Mechanical Engineering (pA)

It will be demonstrated that the student:

I { Disciplinary Knowledge

S 1,_l}.a~ !h~ l<rJ.o.YVle9~e_ o} £h..e!I1~st.ry_ ~n.9 .9<!Ic..ulu~ : l2.a~~d_pl1l:slc~ - I I ( D' ' I' Sk'il

with depth in at least one I I . ISCIP Inary I s

S2,_

~a~

1h..e _api!ity 10_

'Pply

.?~v

_

a!]c..e9

.!l1

_

a~h~l!l~t~~

!.,o

'p~o

_

b~e!!l

~

I / I ,( Disciplinary Skills involving thermal and mechanical systems /

S3,_ ~a~ !h~ .§I~I~y_t~ ~PF!Y ~t.§ltls~c3 .?r:!d}!)~a~ ~19.e!>~a Jo_p!~b!e!!l~ ./

involving thermal and mechanical systems

Means of Assessment (pA)

,

I

,

I

,

These formative/ summative assessments align with Learning Outcome S1,

These formative/ summative

I I assessments align with Learning I I Outcome S2,

S1,_PJ~t~sJJJ~slJ.F§p-ort ... aJ1.9 .tiQI]~"'Lor.k ___ , _________ I I I

"

S2,J:!o.rn~-"/Y,o!k.;. Ie~t §~nLor. Q~sjg[l ______________ ,," These formative/ summative

assessments align with Learning Outcome S3,

Shifting of Program Design Focus:

In Preparation for the

4-Year Degree

Example for Business

Name of Institute: Eastern Oregon University

School of Education and Business: Business Division

URL: http://www.eou.edu/ctl/documents/BUSINESS%20ASSESSMENT%20PLAN. doc

The alignment between intended learning outcomes and assessments is strongest for intended learning outcomes 1 to 4 and weakest for 5.

Outcomes of the Business Administration Degree Program (p.3)

1. Graduates of the Business Administration degree program will possess an integrated knowledge of the functions and systems of business. _______________________________ ....

2. Graduates of the Business Administration degree program will

Disciplinary Knowledge.

"Possess" may be difficult to

measure. Cognitive processes

such as "summarize" or "infer"

could be considered.

have developed strong interpersonal and communication skills ( )

. . Generic Skills

necessary to succeed in the business environment. ___________ - -'--_ _ _ _ _ _ _ _ _ ---'

3. Graduates of the Business Administration degre'e program will have strong technological skills including the use of the Internet,

word processing, spreadsheets, and presentation software. __ .... _

-Generic Skills and Disciplinary Skills. Depending on what is

"used", disciplinary knowledge

may be required e.g. subject

specific spreadsheets 4. Graduates of the Business Administration degree program will

be able to demonstrate the capability of applying business concepts to real world projects. _________________________ _

5. Graduates of the Business Administration degree program will perform effectively and efficiently in business managerial positions. __ ~

, ,

6. Faculty, administrators, and staff of the business division create an environment that fosters excellence in learning.

---,

7. The Business Administration Degree Program provides the

necessary education and experiences that enable graduates to be effective leaders in a global business environment. /

,

/

Disciplinary Knowledge. How

will students "demonstrate" this?

, Is this part of an assessment?

Can "demonstrate the capability of" be omitted?

, ' ( Attitudes

, , These are not intended learning

/ outcomes for students.

I

)

Student Outcomes Assessment (p.4)

1. National Exam - Major Fields Test. Used to assess fundamental .

---,

knowledge of business concepts. Scores will be compared to the "

national average. This test will normally be administered in BA

498 Business Policy and Strategy.

2. Integrated Case Study. Used to assess students' ability to

integrate knowledge and u-s~ ~ to -rn""ak; decisions andto as-s~ss - - "",

" This summative assessment

" aligns with intended Learning Outcome 1.

effectiveness as a member of a team. (Scoring rubric Appendix). '

This summative assessment aligns with intended Learning Outcomes 1 and 2.

The case study will be required for BA 498 Business Policy and Strategy.

3. Senior Project. _ ~o_a.?~e.?~ ~t~d_e!:!t~'_api~t~ ~o_apel~ ~u_s~~s_s ____ _

concepts and the capability of planning and organizing a major

task as well as to assess studentsO oral and written communication

skills and team skills. (Scoring rubrics Appendix). To take place

in BA 490 Senior Project.

This summative assessment aligns with intended Learning Outcomes 2 and 4.

This sustainable assessment

" aligns with intended Learning 4. Technology Portfolio. _

I~ ~

s

_

s

~

s

_

s

_

~

h

~~

h~

r

_

~

t

~

d

~!:!

t

~

'_

h

~~

e

___ ,," " Outcome 3.

advanced technological skills. Assessment will take place in BA ' - - - /

325 Information Management. (Scoring rubric Appendix).

Shifting of Program Design Focus:

In Preparation for the

4-Year Degree

Example for Humanities and Social Science

I

Name of Institute: University of Maryland University College Department: Communication, Arts, and Humanities

Program: Baccalaureate Degree Program in Humanities U RL: http://www.umuc.edu/outcomes/susp/ans/HUMS.pdf

Overall, the alignment between intended learning outcomes and assessments is quite strong for all outcomes except for 1 and 14.

Program Outcomes (p.7):

Upon completion ofthe Baccalaureate Degree Program in Humanities, graduates will be able to:

Generic Skills for intended

• • • • • • • • • • . . . .. " . . . 11 • • • " . : .... 1 Learning Outcomes 1-10.

1. : Use effective oral communication skills to present ideas in both : " "

• public and interpersonal settings : ,," 01" ,

,

2. : Create written communication appropriate for the purpose and

1

, which meets standards of style and grammatical correctness. :

·

·

,

3. : Write analytical, critical, research-based prose that addresses:

; interdisciplinary issues. :

Disciplinary Knowledge,

: • / / Disciplinary Skills

4. : Evaluate technological concepts related to computers and : /

1

components of information systems. _______________ : __ / /

:

.

·

5. : Use online resources to locate scholarly articles and other

: materials for interdisciplinary research.

6. : Address recognized research needs by retrieving, evaluating,

: and using information appropriately.

·

_·

7. : Use primary and secondary sources to conduct interdisciplinary :

l

research effectively.

1

·

:

/ I Disciplinary Knowledge, I Disciplinary Skills

8. : Apply mathematical and numerical reasoning skills. ______

1

_

/

/

• • Disciplinary Knowledge,

·

.

• / Disciplinary Skills

9. : Exhibit quantitative skills necessary for analysis and synthesis: /

: of quantifiable human experience. ________________

1

. _

/ /

·

10. Analyze issues across traditional discipline boundaries. ,

~ ... "

...

" .. ,.

...

"

...

"

...

"

... .

11. Identify key concepts and principles of natural sciences. ______ /

/

/

Disciplinary Knowledge,

I

tI

Disciplinary Skills

12. Analyze the influence of science on the development of

contemporary Humanities perspectives. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Disciplinary Knowledge, Disciplinary Skills

13. Demonstrate knowledge of and appreciation for the traditions and major ideas within the sphere of Asian economic, political, cultural, and historical developments. ________________ ...

14. Integrate knowledge of other cultures to respond appropriately to a varietuy of cross-cultural situations. ______________ _

15. Evaluate current events based on chronological knowledge of various historical events, ideas, and individuals.

_---

_,

16. Use knowledge of other cultures, politics, ethics, and human

rights to positively impact the community, work, or physical

environment.

_---

_,

\

,

"

Disciplinary Knowledge, Attitudes. How would students "demonstrate" this? Would it be ... part of an assessment?

Consider to use alternative words such as "explain" or "describe".

\

\

, Disciplinary Knowledge, Disciplinary Skills, Attitudes

'\ Disciplinary Knowledge, Disciplinary Skills \ \ Methods of Assessment (p. 8-11): 1. Research paper ____________________________ _ 2. Examinations (Course/Chapter) __________________ _ 3. Essay/writing task ________________________ _ \ \ \ \ \ \ \

,

\ \ \ \ Disciplinary Knowledge, Disciplinary Skills, Generic Skills, Attitudes

\ This summative assessment

aligns with Learning Outcome 2.

\

\ \

\ This summative assessment

\ aligns with Learning Outcomes 4,6,8, 11 and 15.

This formative assessment aligns with Learning Outcomes 3,5,7,9,10,12,13 and 16.

eLi

Fifth Round Award8d Proj8cts hav8 88gun!

Ibis is the fifth year of the Continuous Learning and Improvement (CLI) through teaching innovation project. The focus of the fifth round CLI projects is on the

improvement of student leaming. Research (Biggs 2003)

in recent years has shown that to bring about more effective

leaming, students should be encouraged to adopt a deep

learning approach, which can be achieved by such methods as engaging students in authentic learning experience,

adopting student-oriented approach in teaching, using appropriate fOlms of assessments, and setting the students' workload at an appropriate level. In order to evaluate the impact of their teaching innovations on student learning,

CLI project leaders are encouraged to make systematic effo11 to collect solid evidence of improvement in students'

perfOlmance. Twelve CLI projects have been awarded:

one program-level, six course-level and five adaptations. Here's a quick glimpse of the sub-projects and their main objective:

Program Level

Pilot Courses for Science General Education in the Four-Year University Curriculum, Michael Wong (PHYS)

To develop and offer new joint-department General Education (GEE) based on inter-disciplinary themes.

Course Level

Spreadsheet Engineering - The Application of Excel's Macro Programming Language, (Visual Basic for Applications)

to Core Courses within Chemical Engineering, John Barford (CENG) To provide structured teaching of Excel - Visual Basic Application (VBA) in an introductory first year course and application of VBA throughout major core courses during the remainder of the UG degree courses.

Enhanced Student Learning in Civil Engineering Courses Using CAS Calculators, Thomas Hu (CIVL)

To develop teaching materials using a new solution methodology for engineering problems using a new class of calculators embedded with Computer Algebra Systems (CAS).

Assessment Strategies in Problem-based Learning, Mike So (ISMT) To develop assessment strategies to align effectively teaching methods and learning outcomes to assessment in problem-base learning.

A Learning-by-Example Learning Module for Programming Courses, James Kwok (ISMT)

To develop an online learning module delivered through LMES to help business school students to learn their first programming language in a more effective way by means of learning-by-example.

Course Level

Biochemical Basis of Parkinson's Disease & the Therapy, Yi Fan Han (BICH) To develop a 3D animation software package for biochemistry and biology major students to demonstrate the biochemical basis of Parkinson's Disease including the symptoms, pathology and therapy.

Virtual introduction to Biomolecules and their metabolic intermediates, Robert Ko (BICH)

To offer a step-by-step approach to understanding the structures, properties, and functions of major types of biological molecules.

Adaptations

An adaptation is any attempt to modify or enhance an existing innovation, which could be a teaching aid or method, implemented in anyone of the previous CLI projects.

Production of advanced illustrative animations of finite element simulation for teaching in structural mechanics and stress analysis courses, Ricky Lee (MECH)

To generate more advanced illustrative animations of finite element

simulation for teaching in structural mechanics and stress analysis courses. lecture PODca.ting

t ... · r .. r,.F"(t'" ',I. J [I'll I, " ) ( til" , .•• ~ In: t/.I,}IIU1If ';.',J'-,lt,rJ

<l"lj ><,t· .I'u, ,. J'Illt.,>!.,I!P"Ilf,.n Extending GONG to enable audio podcasting for teacherllearner

communication, Jogesh Muppala (COMP)

To adapt the GONG tool to enable bi-directional audio communication

between the student and the teacher with the content available as podcast.

Adapting Gong for LANG208, John Milton (LANG)

VO.lt!;C I

"I.!.lIt,U \'~'H' l_hr~-;C4'_'~!" . "{I"~ IoI",r: .. :_>? .'} ... , ~1 .. ·-•• 1.."<J. ,.. t'~~"'~.'.I·" *"~'!.d I, t...>Mr,IIL"'t'1I'~"'<""".I"'JI-;-...r,I,· ... )l",:-"",1..("

'..t'H ·.¥1lh':' ... P~l·"'.f\'-t'''·H~ .I.)'~r> .,..",. ... ·.1;~ "JJ(.lVJ'O.':Yi'.~~

""".:.,,6)"1'. (,U',_& 4o'.a s.' ' .... t .. 1~· .. ~ ... , J. .. -t"';T"~ '.1 """"'.4

l ".l(""-' ~"'I'~'"'' s,,,',:v ·.4't~·\·· .. ",IoI\'t • .,$.I >t, ... \' •• ~" •• ",)1f_ ~~~:~~~~~r;;'~';,~~:ff!f~~:~;,~:'~(:V:':~'! :S:~::::::'

To produce a simplified audio recorder that can be incorporated into any web page.

Bioinformatics in Bioengineering: Direct Experience with Interactive Learning, Kathy Luo (CHEM)

To redesign the software and website developed in the original project

for students in bioengineering.

Adaption of Podcasting techniques for a postgraduate Computer Science Course, David Rossiter (COMP)

To provide students access to the lecture material via a podcast.

If you are interested to find out more about the projects, please visit http://celt.ust.hk/c/iI.

Tech Camp 2006 Winter

A

new round of Tech Camp, aimed at emi.ching teaching staff's information and communication technology (lCT) skills in a fun and exciting environment, was held again on 11-19 December 2006. With the theme, Click!

Edpedition!, pmiicipants were able to learn a wide range of skills with technologies that support their instmction.

What participants say about Tech Camp?

~ "Useful & interesting"

About 40 teaching staff participated in the five mornings and one whole-day session, they learnt techniques in basic image creation and editing, video production, creating interactive presentations, instmctional websites and e-pOlifolio by using built-in tools including

MS PowerPoint, MS FrontPage, Adobe Photoshop and Adobe Premiere.

"It can really provide me more chances to practise."

"Opportunity to learn more advanced skills, leading to more professional web site design possibilities"

i.) "The practice session was useful and tailored to our own preferences."

j "Thanks for a good workshop with friendly support from the instruction team."

For more infOlmation, please visit http://celt.ust.hklinstriMiniTechCamplhome.html

Spring 2007

12

-Recent Events Highlights

TA Enhancement Program Spring 2007: Teaching Assistant Enhancement Program for new or Year 1 TAs at HKUST started on 25 January 2007. For details, please go to http://celt.ust.hklta!

Tech Camp 2006: Tech Camp was held from December 11-19, 2006. Learning experience incorporating image editing, video production, creating interactive presentations and building up instructional websites was brought to a group of participants. For more infotmation, please refer to Page 12 of this edition.

v

Seminal': Effects of Problem-based Teaching and Leaming Activities on Student Learning: Conducted by Prof Mike So, ISMT, the seminar was well-received. Please refer to

http://celt.ust.hklseminariarchive.htmlfor more infOlmation.