Introduction

This report covers the process of peer review between Dr Christopher Barton and myself. Included in this report are my findings of difficult, interesting and positive aspects of the process as well as the things I learnt about my teaching. Also included are some recommendations about the use of peer review in a University setting.

Review process

Our initial discussion about the peer review process started immediately during the first FFOUT workshop, where Christopher and I scheduled an initial meeting. In preparation for this meeting I compiled a list of items that I wanted to be reviewed on. 
We started the peer review process over a cup of coffee. We exchanged lists and we discussed what lesson and what aspects of our lessons we both wanted to be observed and peer reviewed on. 
Christopher wanted me to review the following aspects of his tutorial:

• Lesson plan
• Handouts
• Structure & delivery of tutorial
• Student/teacher interaction
• Student to student interaction

Christopher also wanted me to informally ask some of the students, regarding whether they feel comfortable now to complete the assessment task.

I wanted to be observed on the following aspects of my FLO Live workshop:

• Lesson Plan
• Learning Outcomes
• Structure & delivery of workshop
• Affects of sitting in front of a PC on student teacher interaction
• Student to student interaction

After having decided which lesson we wanted to be peer reviewed on we agreed on a date and time to sit in each other classes and observe the other person. We agreed that I would be peer reviewed in one of my “FLO Live Workshop” sessions and that Christopher would be peer reviewed in one of his tutorials “Qualitative Research Methods” on 12/05/2011.

Two weeks later we met again to exchange a lesson plan and an outline for our lesson to be reviewed so that each of us was prepared for the role of reviewer.

Christopher sat in on my class and a few days later, I sat in on Christopher’s class. Each of us video recorded our sessions to enable us to complete the self - review of teaching exercise. After our sessions we prepared written feedback (see appendix A and B) for each other and then met again to pass over that feedback and to have a chat over another coffee. We scheduled another date to prepare for our peer review presentation. At this final meeting we went over the details and finalized our part in the presentation. Our presentation was well received, and this is due to the planning and commitment of the two of us.

Most difficult, interesting and positive aspects of the process

The whole process of peer review was new to me. The most interesting aspect was engaging with the process itself, and to actually take the time to really think about my teaching, not teaching in general, but what I do and why. 

The most difficult aspect was to fit this additional work on top my existing workload. Another challenge was actually watching myself on video. I found this quite confronting and intimidating, although, these initial feelings reduced over time. Although this was challenging, it was a valuable perspective to be able to view myself as an observer. Without this task being a requirement of the course, it is not something that I would ever choose to do myself.

Working with my peer review partner was one of the most satisfying aspects of this exercise. We had a good collegial relationship and a commitment from the beginning to be supportive of each other. We both come from different subject areas, but because the focus was about the process of teaching, we were able to review each other productively. Just to have someone to serve as a sounding board, or to be required to present my lesson plan to was valuable. I appreciated his objective opinion and feedback and I felt that he was very supportive.

Did you learn anything about your teaching

One of the best opportunities to learn about my own teaching was gained in sitting in as observer and reviewer of Christopher’s class. I found that by watching someone else doing things well, or sharing the same challenges, pointed to similar issues, strengths and weak spots in my own teaching. For example, one of the things that I would like to include more in my own teaching, which I saw Christopher do, was to utilize students to assist in the teaching process. 

I also learnt that there are some things that I do well, and I grew in confidence in my teaching as a result. For example, I have become very clear of the need for accurate and clearly defined learning outcomes. By clearly identifying what it is that I want students to achieve, I know that I am better able to support their learning. I was able to help Christopher to reword his learning outcomes. For example, Christopher used the work, ‘understand’ 4 times in his intended learning outcomes, but how do we measure understanding? By selecting appropriate verbs, (analyse, compare, visually represent) the assessment of the learning outcomes is supported.

I have learnt that teaching evolves over time and that everybody has a different teaching style.

What would you recommend about the use of peer review in a University context

I recommend that the process that was used is very effective in a University setting. Clear guidelines about each stage of the process and the activities to be completed at each stage were essential. Having these sub stages broke a large process into smaller manageable parts, which was important for motivation and time management both important factors for full time employees. Regular meetings with the peer review partner are an important part of the process. I also recommend the random allocation of partners. At the final presentation it was clear that the process had not worked so well in some case. I think it is important that both reviewers need to be clear not only on the process but also on what exactly they want to be reviewed on.

Conclusion

To conclude, the peer review was a very effective and valuable exercise not only to uncover weaknesses in the way we both teach, but also to discover our strength. My initial fears about being in the spotlight and having my teaching observed and reviewed proved to be unfounded. The fact that my peer review partner came from a different subject area mattered little, or in fact, not at all, as both of were reviewing the process of teaching rather than subject matter content. I was great to have the objective support of a reliable peer review partner.
 

Appendix A

Observations and feedback for Christopher Barton – Qualitative Research Methods (Tutorial, 12/05/2011)
The aim of this tutorial is to provide students with the skills to carry out a thematic analysis in class using one of the following approaches, 1) block & file, 2) conceptual maps; or 3) hierarchical coding. The tutorial provided the basis for students to complete the next phase of an assessment task, to conduct a thematic analysis. The students came from various areas of Health Sciences. Attendance of the tutorial is compulsory. The tutorial session was scheduled for 60 minutes, but finished 10 minutes early, this allowed a number of students to approach Chris and ask individual questions. 

Chris and I met two weeks prior to the tutorial to discuss which aspects he wanted feedback on:
• Lesson plan
• Handouts
• Delivery of lesson
• Student teacher interaction
• Student to student interaction

Chris wanted me to informally ask some of the students, regarding whether they feel comfortable now to complete the assessment task.

Lesson Plan

The lesson plan provided is comprehensive it also included timelines for individual activities. I was very impressed by how smoothly the session ran and how much was covered in the lesson. Chris covered all aspects in the lesson plan in the allocated time. One aspect where Chris could improve the lesson plan is the learning outcomes, by being more specific. For example the word “Understand” is used in four of the six learning outcomes. How do you measure “understanding”? This question is addressed by John Biggs’ work, which he called the SOLO taxonomy (Structure of the Observed Learning Outcome). I have included a link to John Biggs website. There you will find a list of useful verbs, which also indicate different levels of understanding from simple to more complex. http://www.johnbiggs.com.au/solo_graph.html 

Handouts

The handout that Chris provided contained a detailed overview of the activities that were to be conducted during the tutorial. The activities were based around four case studies that Chris used as an example on how to approach the block and file analysis.

Delivery of the lesson

After completing a role call to check attendance Chris started off by outlining the aims of the lesson and going through the intended learning outcomes. The students were asked if they had looked at the assessment task. The majority replied that they hadn’t. To provide a context for the analysis, Chris asked the students to read the four case studies that were on the handout. Once the students had read the four case studies Chris asked the question “What did you think were the key issues?” Overall students showed a good amount of interest. Chris clearly demonstrated using the whiteboard two approaches on how students could go about doing the analysis, either via tables or a conceptual map. I really liked the idea of Chris getting the student actively involved. One of students volunteered to write down some of the findings on the whiteboard. This allowed Chris to focus on his teaching and to continually interact with the class. By showing the students these two possible approaches students could choose the one that suits them best for completing their assessment task. I thought this was a good approach as this facilitates for different learning styles; for example the more visual learners are probably more comfortable with developing a conceptual map, whereas the read-write learners could use tables to present their findings. By letting the students choose which approach suits them best to solve the problem the whole activity becomes more student centered. 

Chris went through a series of exercises with the students and the students were clearly engaged and showed interest. For example, a significant number of students asked clarifying questions, how to go about the analysis. I also noticed that while Chris was clearing the white board a number of students were talking to each other about the task. I did observe a generous amount of student/teacher interaction and student-to-student interaction, which are both indicators that students are engaged. I suspect that one of the reasons for this is the fact that the lesson provided the basis for students to complete an assessment task. 

I did meet with some of the students after the tutorial to ask them how they felt about doing the assignment now and whether the information Chris provided and the examples he gave allowed them now to comfortably complete the assessment. All replied they were happy with what Chris had demonstrated and the examples he had given. One student replied he is reasonably comfortable, this student also noted that he doesn’t care for the topic.

In summary, this workshop was taught very well. The activities that were completed during the lesson were relevant to the students and linked to an assessment task. Students perceived the tutorial as most useful and of value to their study.
 

Appendix B

Observations and feedback for Christian Ohly - FLO Live Workshop 03/05/2011
This workshop is provided as an introduction to FLO Live for the use in a teaching context. All the attendees were academic staff of Flinders University, but from a diverse range of disciplines. The workshop lasted 2 hours, as planned, and compromised presentation of information by Christian, demonstration of use of FLO Live, and then replication of this demonstration by participants.

Lesson Plan

The lesson plan is straight forward and includes key learning outcomes. One aspect that could have been added is timings for each element.

The session itself covered all aspects of the learning outcomes, however, there was one outcome I thought could be expanded. This relates to planning a FLO Live session. Christian’s presentation only dealt with technical aspects. It might be useful to provide an example of a lesson plan, or a real world example of how FLO Live has been used in teaching, that others could use as a starting point for developing their own FLO Live teaching plans and teaching sessions.

Handouts

The handout consisted of a copy of slides from the presentation. This is one aspect that Christian could perhaps utilise further. For example,
• Students were taking notes on the handouts, but there were some points where there was information that students wanted to record, but no obvious information in the handout to record against. It may be useful to include some additional slides (pages of the handout) that have key points that you talk about during the session (e.g. discussion about audio settings).
• One student asked for a ‘cheat sheet’ or a copy of a manual for FLO Live. This seems like an excellent idea to provide to all students for them to take away.

The session itself

Overall, the session was conducted to a very high standard. I didn’t observe any obvious problems in the interaction between Christian and the students, nor, between the students themselves as a result of having computers on their desks. The positives of Christian’s teaching were his enthusiasm, his use of clear explanations without technical jargon, the has on activities and the offer to ‘sit in’ with the staff members when they came to utilise FLO Live in their teaching.

I had an opportunity to speak with some students at the conclusion of the workshop and the feedback was very positive, and the students could see immediate applications for their learning.

The following comments are minor, but might provide some opportunities for future development of the workshop:

• Some students had trouble logging in at various points, which wasn’t picked up by Christian. For example, for the section on setting up a FLO Live page, 2 students were unable to log in. Surprisingly, they didn’t ask for assistance from Christian (unclear why as there was an opportunity to speak up/ask for help – perhaps fear). One student gave up quickly and just watched the demonstration on the projector. The other student continued to try to log in through different avenues. Eventually she gave up, but was distracted and missed this part of the learning experience. It might be useful to spend some additional time behind the students computers, particularly before key components of the presentation, just to ensure they are all in the right place.
• Discussions amongst group members seemed to be useful adjunct to Christians examples provided real world applications and potential problems in the use of FLO Live. Perhaps this could have been enhanced.
• There was a technical problem with the audio recording aspect of the session. While unexpected, it might pay to check that equipment is working as expected prior to the beginning of each session – although admittedly, technical failure like this is probably such a rare event that it may not be worthwhile?

In summary, overall this was a workshop that was taught very well and seemed to be very well received by students. The setting and the use of computers in the workshop did not interfere with the interactions between teacher and students, or between students themselves. There was a good mix of activities, the pace of the information presented and Christians enthusiasm and ability to engage with the students kept the material and presentation of information interesting and aided student learning about the use of this technology.