The project is about deepening our ability to describe and report on a full curriculum that we may become better at designing and teaching within them. So that students have the experience we intend from our designs and contributions. Below follows the accepted application for the grant.

UniSA T&L Development Grant 2012

Teaching & Learning Development Grant Application 2011

‘Clarifying Curriculum Design Requirements: A Curriculum Mapping Tool’

Principal Applicant (PA):     Dr. George Bradford, Learning and Teaching Unit

Project Team Members:      Dr. Carol Grech, Acting Head of School (Academic), School of Nursing & Midwifery; Dr. Colleen Smith, Associate Head of School: Academic, School of Nursing and Midwifery; Dr. David Birbeck, Learning and Teaching Unit

Reference Group:                 Professor Esther May, Dean: Health and Clinical Education; <Empty>, Acting  Head of School (Academic), School of Nursing & Midwifery; Professor Margaret Hicks, Director Teaching and Learning; Associate Dean (Nursing and Midwifery) Karen Flowers RN, Ph.D. Australian Catholic University; Ms LohLan Lee, Project Director, PCMS.

OVERVIEW [top]

The reality of curriculum design is that educational complexity affords what can be called a hidden curriculum.  However, in Australian higher education the focus is increasingly on the rigor and coherency of curriculum design and alignment particularly as accrediting bodies demand education providers to produce evidence of how the intended curriculum is enacted in the classroom. Acknowledging the potential for a gap between ideology and classroom reality provides the context for a project at the University of South Australia (UniSA) to support the Bachelor of Nursing program with design processes, while simultaneously providing a mechanism to surface the hidden curriculum and avoid curriculum creep.

Informed by the field of curricula design, a mapping alignment tool is devised and populated with institutional data. Program directors and course coordinators contribute to course-level design discussions covering implicit themes, health priorities and domains of nursing practice, that flow through the program, while simultaneously identifying pedagogical or technology enhancements made to the curriculum, such as Inquiry-Based Learning and Web2.0 technologies – wikis, blogs, YouTube, virtual classroom, lecture capture recordings, course web sites, and more. Adding this information into the tool permits tracking, analysis, and evaluation across the curriculum to support design intent with delivered reality.

In short, this project will 1) identify data that characterize the visible and the hidden curriculum for the Bachelor of Nursing program; 2) develop prototype tools and processes necessary to capture the data from the Bachelor of Nursing program; 3) capture similar data from a participating project partner to validate data characteristics; 4) analyze the comparative data characteristics for use with other disciplines; 5) identify necessary steps for generalizing process and tools for use in local and international applications; and 6) disseminate findings to audiences within UniSA, at the partnering university, and to national audiences.

BACKGROUND [top]

In 2011, the School of Nursing and Midwifery at UniSA undertook re-writing the curriculum for the Bachelor of Nursing program, and two academic developers (same individuals as participating in this grant proposal) supporting the effort participated in planning meetings with program leadership. From the planning meetings, one identified challenge is the difficulty in tracking program elements (e.g., integrated themes, health care priorities, and domains of practice) that are treated throughout the course of study and which are not specifically identified within the articulation of course-level objectives or assessments. Attaching the needed details into a framework that includes details tracked by the School and the University was discussed and a prototype tool to capture the data was developed over a couple of weeks. The prototype underwent a series of development steps as the concept was refined and a goal to have the curriculum mapping tool (i.e., the prototype) available at the time the re-writing effort was set. As the goal was met by the deadline for this UniSA grant application, the prototype is fully functional. In addition, roughly a third of the Nursing program was entered into the prototype and meetings with academic theme leaders and course writers were initiated to secure details regarding Health Care Priorities, Themes, and Domains of Practice.

One initial technical challenge arose and was initially resolved: course design details provided by UniSA's Program and Course Management System (PCMS) are in an incompatible format that with the first version of the prototype. Some structural changes were made once a suitable conversion utility was identified and tested. (NOTE: The problem with data compatibility arises from the UniSA format that can be described as customized tables in PDF, and which are unreadable as text data. As a result, the table data must be extracted for read-access in a programmable environment that permits further data manipulations.) Additional programming is not expected (nor planned) during this grant project. However, to continue with format conversions of the remaining course design files from PCMS, a software conversion program will need to be purchased and used. (The cost is minor, and the details are included in the project budget proposal.) Format conversion is necessary as the prototype converts course design data into numerical values for analysis and reporting. Therefore, the PCMS data must be brought into the prototype. The Principal applicant has been in discussion with the PCMS team and a future plan is a possible upgrade to PCMS that will store data in numerical form once optimal formats and structures have been identified.  The latter is not an outcome of the project. Instead, such work will be part of a future project.

During the time the prototype was designed and developed, multiple meetings were arranged and demonstrations of the technology were provided. Attending these various meetings were the following:

• Course coordinators and course writers for the Bachelor of Nursing program
• Dean of Health Sciences, Esther May
• The Online Academic Developer Team
• Leadership staff for the Learning and Teaching Unit, Margaret Hicks and Gavin Sanderson
• learnonline leadership and project team members Paul Sherlock, LohLan Lee, Karl Sellman, Richard Lamb, Wayne Pedder, and other staff attached to PCMS development
• Presentations were also given to Associate Professor Neil Murray of the LTU, Dean of Engineering, Brenton Dansi, and to academic staff in CSI.

It is clear that initial dissemination and feedback have taken place.

AIMS [top]

The project’s principal aims include identifying data characteristics and validating an analysis approach to inform future design characteristics of the program and course management system, and disseminating both the approach and data characteristics to the broad community in higher education through conference presentation and journal article publishing. This grant will be used to include participation of another university to improve validity to the data characteristics, the collection process, and analytic methods to derive findings. These aims can be further described as six project objectives:

• Develop a process for conducting program curriculum mapping
• Identify critical characteristics specific to particular discipline domains (e.g., nursing) and develop the mapping process to fit identified informational requirements
• Adjust and evolve a tracking tool to support derived informational requirements
• Conduct a program curriculum mapping process on a minimum of two programs, one from University of South Australia, and one from Australia Catholic University (ACU)
• Evaluate the process, the informational requirements, and program design characteristics to improve curricula design considerations
• Disseminate the results through journal article publication and conference presentation, as well as to produce information requirement specifications to improve the university system PCMS (i.e., Program & Course Management System) features.

ADDRESSING SELECTION CRITERIA [top]

An identified area of priority for practice improvement in teaching and learning is curriculum design methodology. The project aims will provide tools and approaches to improve curriculum design – either as a planning exercise, an approach to analyze and evaluate existing programs targeted for improvement, or as a means to produce evidence necessary for program accreditation. Within the literature on curriculum design, mapping, or identifying the hidden curriculum, this project builds upon and extends the work of other research in a variety of contexts and geographies.

DEMONSTRATED NEED - A review of the literature [top]

Kopera-Frye and Mehaffy (2008, p. 9) argue for curriculum mapping to improve scholarly teaching and learning through alignment, and if alignment is missing, the identification of necessary adjustments. “A curriculum map also helps guide program design and improvement, including consistency, fairness, quality, and effectiveness (Matveev, Okala, & Cuevas, 2006).” The authors note that curriculum mapping can be used to meet two different needs: 1) examine a curriculum at its design stage or creation (i.e., what they term “Front-end analyses”); and 2) refine/evaluate an existing curriculum (i.e., “Back-end analyses”). An example of front-end analyses would be a case study seeking to answer the question, “Did the research [course] sequence reinforce key research principles? Did assignments fit the skill level of students to minimize anxiety and maximize learning?” (Kopera-Frye & Mehaffy 2008, p. 11). Back-end analysis includes identifying the sequence of outcomes across course numbers and to look for instances of redundancy or overlap of course objectives, artefacts, and course content.

Ozolins, Hall, and Peterson (2008) focus on identifying hidden curriculum and informing and improving curriculum design. Hidden curriculum is described as including unofficial expectations, unintended learning outcomes, the construction of social relations, and how students construct “the hidden curriculum as they respond to formal statements about what is expected of them” (Portelli 1993 as reported in Jaye, Egan, & Parker 2005). The authors note there is little discussion about the informal learning that occurs between students, and how this helps them address the formal curriculum. According to Prideaux (2003), curriculum has three components: 1) what is planned for students, 2) what is delivered to students, and 3) what students experience. From their study, researchers found students could not clearly distinguish conceptually between an “informal” and “hidden” curriculum, but students find value in the informal or hidden curriculum for “…providing ‘richness’ and depth to their learning and for acquiring the skills needed for lifelong professional learning” (p. 608). The authors also find that students consider their informal learning experiences as very important in the preparation for and achieving of success with exams. The authors note that while constructive alignment of the curriculum, teaching methods, assessment, and constructivism promote deep learning (Biggs 2003), they also add Dearn’s perspective (2004, p. 6) to remind the reader that outwardly apparent design may not tell the whole story: “…even a fully aligned curriculum would still fail to address fundamental issues related to the hidden curriculum.” This perspective further emerges as students in the study identify a mismatch and imbalance between assessments’ associations to the formal and informal curriculum that will be significant to their learning. The authors conclude that further work to explore how hidden and informal curricula might influence student learning is needed. Finally, a call is made for an exploration of the strategies students use to cope with the hidden and informal curriculum to identify what is hidden that would seem to be critical to their academic success.

Highly relevant to this project was a similar effort in 2005 to map the Bachelor of Veterinary Medicine and Surgery (BVM&S) program across all years at the Royal (Dick) School of Veterinary Studies at the University of Edinburgh. Bell, Ellaway, and Rhind (2009) clarify their purpose of the project was for “…facilitating curriculum review, improving integration across the curriculum, providing a clear and transparent means of demonstrating outcomes for QA purposes, including accreditation visitations, and providing clarity in an increasingly integrated curriculum” (p. 102). The authors note that medical and related professions are increasingly integrated, while losing discrete discipline emphasis – knowledge and skills associated with specific disciplines become lost in the increased complexity of the education. Further, the authors note the difficulty embedding emerging themes into the curriculum, which become design challenges. Not only is the effort of curriculum mapping useful for the four points previously identified, the authors contend students also benefit as sharing a curriculum map becomes useful for students in their understanding of how course and program elements are embedded and supported in their entirety within the full curriculum. The authors identified the stakeholders in their project as being students, staff, prospective students, and quality-assurance and accreditation bodies, and they provide sample questions these stakeholders might wish to be able to ask as a result of the project. In their project, the team initially developed a spreadsheet tool to capture and categorize course elements and the relationships between elements to identify and ensure constructive alignment. These details were subsequently moved into a database, and other web-based tools were designed to streamline data capture. The authors recommend that curriculum mapping as a project is not to be entered into lightly, as dedicated staff will be necessary not only for capturing data, but to sustain the program. Further, the authors effectively argue for the integration of the curriculum map “…with the rest of the online environment and to align it closely with the program’s culture and dynamics” (Bell, Ellaway, & Rhind 2009, p. 104). The authors conclude their curriculum mapping approach demonstrates its value for curriculum management, review, and development.

In a study to map 31 courses in a Pharmacy program at the University of Arizona, Plaza, Draugalis, Slack, Skrepnek, and Sauer (2007) conducted a mapping process that included three data sets: expected educational outcome details derived from reviewed syllabi from the 2004-2005 academic year (this information was entered into a document titled “Outcomes Expected of Graduates of the Doctor of Pharmacy Program” or more succinctly, “Outcomes Expected”); student ratings of domain coverage identified in the Outcomes Expected document; and instructor responses to a questionnaire on if they teach to competencies identified in the Outcomes Expected document. The authors follow Porter’s (2001) topographical maps to display the curriculum to indicate any relationship between competencies and knowledge and skill domains. To facilitate comparative analysis, the authors conducted some data conversion to permit uniform presentation (e.g., translation to numerical data, proportions, and means to proportions). The authors note that while their methodology to map the curriculum was free of recall bias (Litaker, Cebul, Masters, Nosek, Haynie, & Smith 2004) from participating academics and students, there was no way to determine the extent particular knowledge or skills are reinforced. The authors also note the study’s major limitation was that the intended and delivered curricula were the same. This bears on the validity of the analysis since student responses would reflect delivered curriculum and academic responses to the questionnaire would reflect the intended curriculum, so while similar language was used in each data set, responses would reflect different experiences. The authors highlight the need to rectify the inability to differentiate between the two experiences but do not offer how this might be achieved. However, the authors do note that using graphical curriculum maps to identify potential redundancies or excessive review, and even potential areas in the curriculum that are hidden is useful.

Following the work from these studies, the proposed project similarly recognizes the general need to be able to identify and track knowledge and skills across a curriculum with the more specific need to extract and quantify overlap, redundancies, missing, or hidden knowledge and skills. Further, one area the published projects fail to address adequately is the alignment of educational strategies to targeted program outcomes. Developing tools and processes aimed at capturing such information is expected to provide strong insight into strengths and weaknesses of course and program design. Indeed, this is a core part of the project.

Additionally, this project follows Kopera-Frye and Mehaffy (2008) and Bell et al. (2009) to permit program design support, as well as program design evaluation. Both strategies are centrally important to schools and universities in their efforts to build new programs, while also being able to study the effectiveness of existing programs. Further, the project will take a stronger approach than Bell et al. as the aim is to leverage easily developed technology (e.g., spreadsheets) to inform design requirements for robust database analysis systems, such as PCMS. Where Bell et al. note the need to place curriculum data in a formal database system, they acknowledge the level of effort required to build and sustain such a system is beyond the means of many institutions. However, at UniSA, a system is in place (i.e., PCMS), but the missing components are validated data types and a data processing approach (i.e., that includes data capture, analysis, and reporting) that provide curriculum mapping to fulfill institutional needs, similar to the needs as articulated by Bell et al (2009).

In recent years, two grants have been funded by the Australian Learning and Teaching Council (ALTC). Professor Bev Oliver (Curtin University) through a fellowship grant “Assuring graduate capabilities: evidencing levels of achievement for graduate employability” (ALTC 2011) created as one component an Excel spreadsheet tool she refers to as “CCMap”. Dr Romy Lawson (University of Technology Sydney) leads the ‘Hunters and gatherers’ project which maps graduate attributes and collects assurance data (Lawson, Bajada, & Lee 2010). It must be said that in both instances, there are a number of similarities, which is inevitable given that for curriculum development and analysis we seek to achieve many of the same outcomes. For example, in terms of mapping assessment types and aligning course objectives to assessments, all the tools look and do essentially the same thing. Attribution and acknowledgment need to be given to Dr Lawson, as an early version of the Hunters and Gatherers software, shared by Dr Lawson, was important in the very early discussions about the UniSA tool.
A rough comparison between the proposed UniSA prototype to Oliver’s CCMap follows. CCMap supports curriculum analysis across six, well-developed, areas (Oliver 2010b) (Note: below “course” refers in UniSA terms to “program/ unit”, which is essentially a UniSA “course”):

• Course Analysis related to Unit Learning Outcomes
• Course Analysis related to Assessment
• Course Analysis related to Learning Experiences
• Course Analysis related to Learning Resources
• Course Analysis related to Curriculum Themes
• Course Analysis related to Career Development Learning 

The UniSA tool differs from CCMap in a number of ways, but there are two major differences. Firstly, area 1 in CCMap simply collects numbers of reference points and indicates whether or not Graduate Attributes are obtained, as well as other more problematic data relating to the use of specific verbs across the program as a kind of indication to the degree of mental effort (or “thinking”) based on Anderson and Krathwohl’s revision to Bloom’s Taxonomy of the cognitive domain (Oliver 2010a). The proposed UniSA tool does this as well, but to a much greater depth in being very explicit about Program Objectives aligning to Course Objectives. This is a much tighter fit as courses are then conceived as contributing to the attainment of explicit Program Objectives.

Secondly, regarding course analysis related to themes, Oliver’s CCMap collects data against assessed themes. However, the UniSA prototype is designed around the idea that not everything needed in a curriculum is and can be assessed. For example “quality and safety’ is a theme through the nursing curriculum, which every course needs to develop, yet not all courses will specifically assess.
The project team is familiar with Lawson’s Hunters and gathers tool, as Romy was kind enough to show us her technology earlier in 2011. Similar to the UniSA prototype, it has a tighter link between Program Outcomes and Course Objectives than does CCMap but lacks, like CCMAp, the ability to map the hidden curriculum (i.e., course elements that are not assessed). The Romy technology is very much about assurance for learning: when you can sign off a Program Objective as “done”.

Further, the UniSA prototype is designed to track more course elements to later permit an effectiveness analysis of mediators (i.e., course elements) to learning, such as employing particular teaching and learning strategies (e.g., Inquiry-based Learning), technology enhancements (e.g., ePortfolio, LRS, Wikis, Blogs, etc), and formative assessments. Formative assessments can be linked to Course Objectives as well. There is the possibility of entering scheduling data associated with summative and formative assessments to provide program-wide analysis of academic load to facilitate curriculum planning. Each mediator, or course element, can be described with text to explain, for example, the rationale for its inclusion. This can later be revisited should assumptions or conditions change, and the rationalizations require revisiting for validity.
In summary, the UniSA prototype shares some features, but it also captures to greater depth the course elements being employed and the relationships between major drivers Course Objectives, Course Assessments, and Program Outcomes. Most importantly, the prototype permits tracking of elements that would otherwise remain invisible because they are not assessed, such as discipline-specific priorities, themes, domains of practice, or similar content areas.

TARGET OUTCOMES, ANTICIPATED IMPROVEMENTS, MEASUREMENT STRATEGIES [top]

The project aims to validate approach and data types to analyze curricula. This information will be used to support future development of a university system that tracks details associated with course and programs. Further, this project will disseminate the approach and data types to the broad educational community. Through the approach to improve data validity, this project will collaborate with at least one additional University. By working with another University, the collected data will represent a more fully authentic sample.

The anticipated improvements to result from this project include an increased ability to track the details in a program that can be lost within the complexities. Following the terminology of Kopera-Frye and Mehaffy (2008), this project will yield a tool and an approach to facilitate conducting either front-end or back-end analyses for program design or evaluation. This project will also facilitate how Program Directors can respond to requests to provide evidence where and how varying knowledge, skills, or enhancements, such as technology or pedagogy, are embedded across a curriculum.

One specific target outcome of this project is to favorably improve the features and functions of UniSA's Program and Course Management System (PCMS). With regards to course and program reporting and analysis, the current version of the system offers basic functions that permit inquiries to generate reports on administrative details relating to courses and programs. To be useful for educational research and analysis, a deeper level of detail needs to be added into and tested with the prototype. Additionally, some programs include program outcome targets that are not tracked, as this information is not articulated within the objective statements of assessment details. The prototype is designed to store this information and produce reports that have a multitude of uses, including but not limited to the alignment between objectives, assessments, program outcomes, and a diverse set of design elements such as discipline-specific priorities, themes, and domains of practice, as well as technology-enhancements, formative assessments, and rationales for inclusion. Part of the scope of the project is to identify and develop reports that the School of Nursing will find useful for reasons, such as reporting how changes to an element of a course such as an assessment item will impact on the overall coherence of the program. Ongoing discussion will continue with the PCMS Development Team about this developed and tested prototype with the potential to inform future upgrades of PCMS.

One potential application of this level of data and formatting of information will be to permit research into course and program design effectiveness. Currently, such research is not possible without manually assembling the information, converting the data into workable formats, and analyzing student retention and academic achievement against that data. The long aim is to direct the development of PCMS to permit such analysis. This type of analysis will not be restricted to the School of Nursing but can be extended into other disciplines. The directing philosophy for the project is to identify data structures and formats that will scale to permit University-wide application.
Curricula that have accreditation requirements including reporting responsibilities (e.g., Bachelor of Nursing) have complexities that are different to non-regulated professional programs.  For instance, within the Bachelor of Nursing, the accreditation body requires demonstration of a clearly aligned curriculum that shows a scaffolded approach to the development of integrated themes, national priorities, skills and domains of practice.  The linear nature of current curriculum design makes it difficult to demonstrate the horizontal and vertical alignments within curriculum.  This mapping tool will make this possible and opens the door to a new level of analysis, evaluation, and reporting. The simplified alignment approach of assessments to objectives will need to be re-addressed within this context. Future work will likely expand to improve the data mapping technology, as well as explore other disciplines at a national level.

APPROACH [top]

The approach includes the specification of details to manage the activities and resources to complete the proposed project within the time specified and the estimated budget. Therefore, the approach includes the following subcategories to describe how this project is to be managed: Project Scope – presents scoping statements and limitations; Project Stakeholders – identifies key groups or individuals who have a stake in this project; Project Timeline and Milestones – identifies the schedule and set of deliverables; and Risk Management – identifies the key factors that may negatively impact the successful completion of the project.

Project Scope [top]

The scope of the project includes the six project targets as described in the AIMS section. This project is limited to the Division of Health Sciences, while the project team recognizes the potential for future application in other Divisions. Accordingly, resources available to support the project are limited to Division of Health Sciences and the City East Campus Learning and Teaching Unit (CE LTU) team members. Further, this project includes inviting participation to at least one outside institution (ACU) to provide validity to results and findings. The budget and timeline for the proposed project is the limiting factor to include only one outside partner.

This project is also limited to a single discipline to support the target of validating the approach and derived informational requirements: the inclusion of different disciplines would increase the number of variables and decrease the validity of results. The selected discipline is undergraduate nursing and the program of study will be the Bachelor of Nursing & Midwifery, since this is the program that initiated the project conceptualization. As the School of Nursing and Midwifery at UniSA already holds strong ties to the Australian Catholic University (ACU), the scope of this project is to include ACU as the collaborative partner. Associate Dean (Nursing and Midwifery) Karen Flowers RN, Ph.D., has been approached and has committed to participating in this project.

Project Stakeholders [top]

Project stakeholders for this project include the following groups or individuals:

• LTU – UniSA: Professor Margaret Hicks, Director LTU and PCMS Owner; Associate Professor Gavin Sanderson, Deputy Director LTU and Team Lead to project participants
• Chancellery: Professor Joanne Wright, Deputy Vice Chancellor & Vice President: Academic
• PCMS Development & Support Team: Lohlan Lee, and others To Be Determined (TBD)
• Division of Health Sciences – UniSA: Esther May, Dean: Health and Clinical Education; <Not yet appointed>, Acting Head of School: School of Nursing and Midwifery
• LTU – ACU & Division of Health Sciences, School of Nursing and Midwifery: TBD

This project has the potential to serve many in the University. Therefore, the stakeholders provide counter-balance and University-wide perspective to the project team to ensure that the outcomes not only serve the project team and the immediate needs of the School of Nursing and Midwifery, but also to the wider audiences at UniSA. The level of involvement expected of the Project Reference Group includes attending to communications (e.g., emails) and meeting demonstrations with discussions. The estimated time commitment is 2-3 hours per month.

Carol Grech will remain as a member of the project team throughout the time the project is active. Carol is currently acting Head of the School of Nursing and Midwifery until the new Head of School is appointed. The requirement that the Head of School be a member of the reference group will be met when the new Head of School is appointed.

The identified Project Team have a complementary and extensive set of skills, knowledge and experience to complete this project and, as previously indicated, have a well established working relationship with key staff at Australian Catholic University.  An invitation will be extended to Dr Cath Hall to join the Reference Group which will provide an avenue of broader consolation and collaboration related to the millstones and outcomes of the project.

Project Timeline & Milestones [top]

The project timeline comprises process steps mapped strategically to bring the project to a successful conclusion. Process steps and milestones should not be confused with each other. Milestones represent key dates by which specific activities or events take place. The dates are “key” because they represent critical deadlines set to keep a project on target. While it is possible that a process step and a milestone occur on the same day, not all process steps are milestones since they may be adjusted without affecting critical deadlines. The concepts and terms are not interchangeable.

The timeline is structured around a systems development model that is based on rapid prototyping – roughly design, test, design, implement, evaluate, and finalize. The process steps are titled to convey the major effort that describes a particular step, but other efforts are underway simultaneously. For example, as the curriculum mapping tool is being populated with data that will be part of the study, the tool is simultaneously being re-designed to fit needs. While “design” is mentioned twice, this reflects that there will be two major design process steps: one for UniSA and one to support the partner university. The “test” process reflects activities to ensure proper tool functioning. The “implement” step recognizes that populating the tool for the partner university may include a number of additional sub-steps to support data population. For example, if the partner university does not have similarly well-articulated objectives, some strategy may need to be developed to provide a work-around. The “evaluate” step includes both analysis of collected data, as well as an evaluation of the approach to conduct this type of work.

The “finalize” step reflects the production of final reports, as well as dissemination through article writing and conference presentations.
Anticipate timelines for the process steps will be the following:

• Design: October – December 2011
• Test: November – December 2011
• Design: January – March 2012
• Implement: February – April 2012
• Evaluate: March – May 2012
• Finalize: April – September 2012

Regular progress reports will be disseminated on a monthly basis to project stakeholders: Margaret Hicks, Gavin Sanderson, Carol Grech, and LohLan Lee from the PCMS development and support team.
Milestones for this project include the following five:

Milestone 1:  Project Proposal and Grant Submission
Target date:  Friday, October 21, 2011, 4pm / Revised date: Monday, January 16, 2012, 5pm
Details: Audience: Project Team, Project Reference Group, and Grant Review Board
The deliverable for this milestone is this document. The purpose of this milestone and deliverable is to kick-off the project with disclosure of the project details and formulation of communication and resource needs, and how these will be derived and adjusted through the work of project team members during the project’s schedule.

Milestone 2:  UniSA Data Entry Complete
Target date:  On or before Friday, December 16, 2011, 4pm/Revised date: February 24, 2012, 5pm
Details: Audience: Project Team, Project Reference Group, and Project Stakeholders
The deliverable for this milestone is the completed entry of course and program design data from UniSA sources into the tool.

Milestone 3:  Partner University Data Entry Complete
Target date:  On or before Friday, April 6, 2012, 4pm
Details: Audience: Project Team, Project Reference Group, Project Stakeholders, and Extended Stakeholders at Partnering University
The deliverable for this milestone is the complete entry of course and program design data from partner university sources into the tool.

Milestone 4:  Analysis and Evaluation Complete
Target date:  On or before Friday, May 25, 2012, 4pm/Revised date: June 29, 2012, 5pm
Details: Audience: Project Team, Project Reference Group, Project Stakeholders, and Extended Stakeholders at Partnering University
The deliverable for this milestone is the documentation of the analysis and evaluation results of the project. Further, the basis of the documented results is used to present at a conference  (14th National Nurse Education Conference, Perth, Western Australia) and to prepare and article for publication.

Milestone 5:  Project Complete
Target date:  On or before Friday, June 29, 2012, 4pm/Revised date: September 28, 2012, 5pm
Details: Audience: Project Team, Project Reference Group, and Project Stakeholders
The deliverable for this milestone is the completion of internal presentations and associated documents to Project Stakeholders and Reference Group.

Intellectual Rights and ACU Involvement [top]

The legal framework regarding the prototype and other supporting resources for this project will be that UniSA will retain all intellectual rights. ACU contributes personnel to support mapping and gets their Bachelor of Nursing program mapped. They do not contribute to project costing because they will not receive intellectual rights to the technology.  

Staff from the Australian Catholic University (ACU), specifically Assoc. Prof. Karen Flowers (Associate Dean of Health Sciences: Nursing and Midwifery); Ms Paula Williams (Faculty of Health Sciences e-Learning Coordinator); and, Ms Natalie Gamble (Faculty of Health Sciences Project Officer: Learning & Teaching Enhancement) have collaborated with the UniSA School of Nursing & Midwifery since early 2010 regarding undergraduate nursing curricula design and inquiry based learning (IBL).  Three members of the project ream (A/Professors Carol Grech, Colleen Smith and Dr David Birbeck) have conducted IBL workshops for ACU and in October 2011, Paula Williams and Natalie Gamble met with the Project Team at the City East campus to discuss, among other collaboration activities, this proposed curriculum mapping project.  As A/Prof Karen Flowers is currently on leave from ACU (returning in the New Year) a formal agreement will be developed through UniSA Contracts Execution and Grants Management Services and submitted in February 2012.

Ms Paula Williams (ACU Faculty of Health Sciences e-Learning Coordinator) and Ms Natalie Gamble (ACU Faculty of Health Sciences Project Officer: Learning & Teaching Enhancement) will contribute to data entry for ACU.

Workload Description for Data Entry [top]

The work of data entry can be generally described as requiring two steps: 1) course details are set up from PCMS-derived data; and 2) deep-level details entered on a per course basis. For step 1, course details set-up, a course worksheet is created by copying a worksheet template from the tool and pasted into a new, blank worksheet. Then, course details are received from the PCMS support team, converted into an Excel format and pasted into the course worksheet. For step 2, the deep details, the work can vary depending on the level of information available (i.e., and can likely be derived from course coordinator interviews or investigation into existing resources). In essence, the details are entered into specific areas of the worksheet. The information may be pasted or manually keyed. In many instances, a check box approach is used. This step will be the most time consuming, as it will be dependent upon the level of detail obtained. The two steps are repeated for each course in a program.

The project will be led by Dr George Bradford who has extensive experience in instructional technology including supporting technology-enhanced learning (TEL) development and evaluation. Members of the project team will work with the project leader and the partner university to ensure the relevance and quality of the data that goes into the prototype, involved in the testing analysis and evaluation of the design process.  A research assistant will convert the PCMS data and work with Course Coordinators from UniSA and the Partner University to populate data into the mapping tool.  This person will also regularly liase with the project team, leader and reference group, and organise the delphi component of the mapping tool evaluation.

Risk Management [top]

Managing risk associated with the project can be reduced to identifying mitigating strategies for three critical areas of the involved work: curriculum development, programming in the prototype technical environment, which currently is Excel, and relationship management with ACU. Firstly, curriculum development expertise is not at risk as each project team member has experience with it, so the loss of any individual during the project lifecycle will not place a successful conclusion at risk. Secondly, Excel programming expertise is held by a single project team member, George Bradford. While the loss of the expertise does pose a risk, the risk is relatively small given that the extensive programming is already complete, leaving an expectation for minor adjustments only.

The Excel environment was intended only for prototyping. The development of future technical solution will lie in a different programming environment, and in this way, Excel programming expertise is not critical: future developers only need to gather what data is being stored; how that data is formatted; what relationships are made between connected data variables; and what reports are found to be useful and which variables they are derived from. These details will be mapped out of the Excel programming environment and into another, such as PCMS. Should Bradford for any reason no longer be a member of the project team, his research interests lie in this area of curriculum development, analysis, and evaluation, so he will remain engaged with the project howsoever. Thirdly, the relationship between the Nursing program at UniSA and ACU has been developed over time with coordinated visits and meetings covering a variety of different goals. While the strongest relationship is managed by Carol Grech, each of the other project team members knows some of the ACU academic staff as well. The associated risk should be characterized as typical of any project, given the personal relationships involved, and therefore the risk should be considered as acceptable.  

EVALUATION STRATEGY [top]

The evaluation of the project will include a qualitative methodology. As a preliminary investigation into the utility of the prototype, a delphi group design will be developed to explore four aspects of the prototype: 1) the identification of reports (and their formats) that will be seen as useful to the School of Nursing and Midwifery,  inclusive of Partner University, as well as reports that may be seen as useful to other disciplines; 2) establish a priority of reports as regards their utility to the School; 3) envision future reports or data capture and analysis that will extend the utility the prototype can provide; and 4) an open-ended  inquiry into the perceived utility of mapping course details across a curriculum – what is critical and what is not. The rationalization for the delphi group design is to address attendance challenges: it may prove quite difficult to have key representatives available at the same time to permit participation in focus groups. Further, the delphi approach is appropriate: it is iterative and leads to agreed understandings. For this type of analysis, it is presumed that ethics approval will not be necessary.

The delphi group approach in general is conducted asynchronously, usually via email. Participants are advised of communication and response details; each participant may not necessarily know the other participant identities. Participants respond to an instrument (e.g., a questionnaire) anonymously, and their responses are revised following a pre-set protocol. The revisions are then assembled into a revised questionnaire, and the participants are again asked to consider and respond. It is possible a third iteration may be used. The final data will then be analyzed for emergent themes and priorities that will be used to direct future work beyond the current grant. The evaluation analysis will not be extensive, but rather indicative: follow on work should include a much more extended analysis and evaluation. The proposed qualitative analysis should indicate where further work will be necessary to permit future quantitative analysis, while also establishing general utility. The scope of this project proposal is to provide proof of concept, not a finished product.

The aforementioned evaluation strategy seeks to build the future case for exploring how courses and programs can be described as effective when considering their design characteristics. The use of the term effective refers to the statistical results of student retention and achievement. To conduct such explorative research, a follow-on project focus will be on differences or relationships of course or program design characteristics against student retention or achievement. This means the independent variable for course or program design characteristics must be identified and isolated – hence the need for consistent data characterizations and curriculum mapping of the program. With such analysis requirements in place, research might proceed by using the Independent-T test to explore differences between UniSA sampled population and the partner university sample population or by using multiple regressions to test for relationships. While these particular tests are not included in the scope of this study, they are mentioned to explain the requirement to evaluate the prototype and data characterizations. For this type of analysis, future ethics approval will be necessary.

DISSEMINATION OF FINDINGS [top]

Dissemination of findings includes internal and external presentations, conference presentation, and expected journal article publication. Internal presentations will be to Dean Esther May, the Program Directors and Course Coordinators for the Bachelor of Nursing program, the PCMS development and support team, and to organizations at the partnering university.

Disseminating information to the stakeholders will include monthly progress reports as emails (possibly including attachments, such as but not limited to Word-based reports and inserted graphics) that will include costs incurred and progress made. These reports will be delivered to all stakeholders / reference group members. Several presentations to demonstrate progress will be organized at key moments in the project lifecycle. These presentations will be given to all stakeholders / reference group members.

Dissemination to the wider University should be through TALC. All four divisions have functional TALCs where this tool can be shown. Further, the dissemination strategy will stretch beyond the University by means of publications and conference presentations, as well as future grant applications. Academic Developers (ADs) will be pivotal in the dissemination strategy around program development. It's important to highlight that the prototype technology is not one you just pick up and use. It is one that needs to be strategically used when it can provide maximum returns for the effort needed. Few programs have coherent program objectives and developing these will be stressful and require long work for any program.

A conference in Perth has been identified and an abstract submitted and accepted by the 14th National Nurse Education Conference being held in Perth, on the 11 – 13 April 2012 (Please see the following website: http://www.iceaustralia.com/nnec2012/).

The benefits from this project are many. As articulated by Bell et al. (2009), the project will facilitate curriculum review, improve integration across the curriculum, provide a clear and transparent means of demonstrating outcomes for QA purposes (including accreditation visitations), and provide clarity in an increasingly integrated curriculum in Nursing. Further, the project will yield useful design information for PCMS and set the stage for deeper analysis and evaluation of what might characterize effective program designs. In addition, the community at UniSA will likely benefit in later iterations as the project is scaled out to support new design and development of curricula across the four divisions, or as an evaluation approach to improve existing designs. Finally, for the community beyond UniSA, at least one partner university will directly benefit in similar ways as the Nursing program at UniSA, and other universities will see an approach to conduct similar projects on their own campuses.

BUDGET [top]

A Research assistant will be appointed for 7.5hours per week for 18 weeks to undertake the duties outlined previously.  No teaching buy-out is required for the principal applicant or the project team members. The travel related costs were provided by Phil Hoffman Travel. Project costing is divided across two stages:

• Stage 1 reflects the cost for data entry and includes two items: 1) procurement of software to convert PCMS generated reports into a format suitable for the prototype (i.e., Excel); and 2) the cost of a Research Assistant to support data entry. Added cost for PDF-Excel Conversion Software: Wondershare (see this URL: http://www.anypdftools.com/pdf-converter-win-buy.html). Single license cost for PDF Converter Pro is $79.95 US or $159.90 US for 2-5 licenses. Costs for the Research Assistant, ARA 5 for 11.5 hours per week for 20 wks ( 6th Feb – June 22nd 2012) @ $41.45 per hour = $6,217.50 plus 16.5 % on costs = $7,243.39   – Total $7,481.31 and a completion date of May 2012 (for details, please see accompanying document, “Staged Budget Pro Forma – HSC Nursing 2011);
• Stage 2 reflects the costs for analysis, evaluation, and dissemination– Total $2,521.00 and a completion date of September, 2012;

Total projected expenses are $10,002.30.

REFERENCES [top]

Australian Learning and Teaching Council (2011), Assessing graduate outcomes: new projects search for a more consistent approach.  Retrieved 20 December 2011, from Australian Learning and Teaching Council http://www.altc.edu.au/April2011-strategicpriorityproject

Bell, C. E., Ellaway, R. H., & Rhind, S. M. (2009), Getting started with curriculum mapping in a veterinary degree program. Journal of Veterinary Medical Education, 36(1), 100-106.

Biggs, J. (2003), Teaching for quality learning at university: What the student does. Philadelphia, PA: Society for Research into Higher Education, Open University Press.

David, P. (2003), ABC of learning and teaching in medicine: Curriculum design. BMJ, 326(7383), 268-270.

Dearn, J. (2004), Behind the formal curriculum. Prof Educ 3(2), 6–7.

Jaye, C, Egan, T, & Parker, S. (2005), Learning to be a doctor: Medical educators talk about the hidden curriculum in medical education. Focus Health Prof Educ 7(2).

Kopera-Frye, K., Mahaffy, J., & Messick Svare, G. (2011), The map to curriculum alignment and improvement.Collected Essays on Learning and Teaching, 1(0).

Lawson, R., Bajada, C., & Lee, J. (2010, 18-19 November 2010), SOS:a tool to support assessment practice across degree courses. Paper presented at the ATN assessment conference: Assessment: Sustainability, Diversity and Innovation, University of Technology Sydney.

Litaker, D., Cebul, R. D., Masters, S., Nosek, T., Haynie, R., & Smith, C. K. (2004), Disease prevention and health promotion in medical education: Reflections from an academic health center. Academic Medicine, 79(7), 690.

Matveev, A. G., Okala, C. G., & Cuevas, N. M. (2006, May), Mapping curriculum maze: A practical model to develop and analyze program curriculum maps. Paper presented at the meeting of the Association for Institutional Research, Chicago, IL.

Oliver, B. (2010a), Benchmarking Partnerships for Graduate Employability (pp. 75): Curtin University.

Oliver, B. (2010b), Curtin Curriculum Mapping Tool -User Guide Version 2.0 (pp. 28). Curtin University: Curtin University.

Ozolins, I., Hall, H., & Peterson, R. (2008), The student voice: Recognising the hidden and informal curriculum in medicine. Medical Teacher, 30(6), 606-611.

Plaza, C. M., Draugalis, J. L. R., Slack, M. K., Skrepnek, G. H., & Sauer, K. A. (2007), Curriculum mapping in program assessment and evaluation. American Journal of Pharmaceutical Education, 71(2).

Porter, A. C., & Smithson, J. L. (2001), Defining, developing, and using curriculum indicators. CRPE Research Report Series Consortium for Policy Research in Education, Philadelphia, PA.

Appendix A: Prototype tool screen captures [top]

Figure 1: Index page presents contents of the file with hyperlink access to content. The Index, Help, Reports, GQs-POs, Template, and Sample pages are fixed. Course pages (see numbered tabs) are added as course information is entered into the prototype. The indexed links to these course pages are automatically added to the Index page.

Figure 2: Reports page presents the initial list of prepared reports that can be generated from the data in the course pages.

Figure 3: Reports page – example report – Program Outcomes to Courses - presents an inventory of which Program Outcomes (POs) are treated in all courses in the program.

Figure 4: Reports page – example report – Courses to Enhancements - presents an inventory of which enhancements (e.g., TELs or pedagogies, such as Inquiry-based Learning) are treated in all courses in the program.

Figure 5: Reports page – example report – Courses to Health Care Priorities - presents an inventory of which priorities (e.g., Arthritis, Asthma, Cancer Care, Cardiovascular Diseases, etc.) are treated in all courses in the program.
Special note: The format of these reports are designed following APA publishing guidelines for ease of reading, consistency of presentation, and for possible inclusion into published papers. Given UniSA follows Harvard Style, these reports could be redesigned following those style-guide conventions; however, this is not within the scope of the proposal.

Figure 6: GQs-POs page presents a centralized resource where program-wide details (e.g., graduate qualities, health care priorities, themes, domains of practice, technology enhancements, specialized pedagogy, etc.) are entered for referencing in all of the Course Pages. The illustrated document mapping GQs to POs is a sample from Nursing that is pasted into the prototype, then coordinate details are entered to the right allowing for nearly automated extraction without need for re-keying.

Figure 7: GQs-POs page – Nursing specific choices – where program-wide details (e.g., graduate qualities, health care priorities, themes, domains of practice, technology enhancements, specialized pedagogy, etc.) are entered for referencing in all of the Course Pages.

Figure 8: GQs-POs page – Group Offering choices – where program-wide details (e.g., graduate qualities, health care priorities, themes, domains of practice, technology enhancements, specialized pedagogy, etc.) are entered for referencing in all of the Course Pages.

Figure 9: GQs-POs page – Enhancement choices – where program-wide details (e.g., graduate qualities, health care priorities, themes, domains of practice, technology enhancements, specialized pedagogy, etc.) are entered for referencing in all of the Course Pages.

Figure 10: Template page contains the space for pasting current details with course information, which can originate as Template 7-type documentation or converted PCMS reports. To the right is the extraction logic that nearly automates taking the details from the forms and converts it into formats that can be used for analysis and reporting.

Figure 11: Template page – part of the logic for data extraction from the different course reports (e.g., Template 7-type documents or PCMS generated course reports).

Figure 12: Template page – part of the area where additional details regarding the course may be entered, such as figured here enhancements and the matching of health care priorities to Course Objectives.

Figure 13: Sample page – demonstrating how pasted-in PCMS course report data is displayed and the extraction settings.

Figure 14: Sample page – demonstrating how pasted-in PCMS course report data is displayed and the extraction settings.

Figure 15: Sample page – extraction and data entry area for course details of summative assessments.

Figure 16: Sample page – extraction and data entry area for course details of formative assessments and relating each to specific course-level objectives.

Figure 17: Sample page – demonstrating how formative assessment details are entered (see Xs to the left) and their extraction as numerical data for later analysis and reporting.

Figure 18: Sample page – demonstrating how nursing specific details are entered, related to course-level objectives, and rationalization or explanatory text may be provided.

Figure 19: Sample page – demonstrating how nursing specific course-level objectives may be mapped to program-level target outcomes.