Online discussions Improving the quality of the student experience

Online discussions: Improving the quality of the student experience
Helen Wozniak University of Sydney
Over the past five years, the School of Applied Vision Sciences at the University of Sydney has used orientation activities to improve the quality of the students’ online learning experience while they are located remotely from the campus undertaking professional practice placements. During 2004 and 2005, students were introduced to Salmon’s five-stage model of online learning and teaching in the orientation sessions to provide a structure for reflection on the quality of their online learning experiences before and after participation in asynchronous discussions. These student reflections were coded and compared to the student ratings of their stage of development as online learners and their academic performance. There was a significant change in the learners’ stage of development and degree of reflection before and after their participation, which was positively correlated with their online participation assessment results. Important insights including strategies that improve the quality of asynchronous discussions are considered.
Introduction
Web-based activities increasingly are being used to supplement learning materials in higher education courses known as ‘blended learning’. A key factor in the way that web-based activities are designed and implemented is a clear understanding of the notion of interaction which Moore (1989) describes as occurring in three ways: learner-to-content; learner-to-instructor; and learner-tolearner. It is the last type of interaction between one learner and another, individually or in groups, with or without the presence of an instructor, which has become an important dimension in distance education in the latter part of the 1990s, and is ideally suited to applications mediated by the internet. As stated recently by Anderson, Annand and Wark (2005), ‘there is a growing body of literature indicating that increased peer interaction can boost participation and completion rates, and result in learning outcome gains in distance education courses’ (p.223). Learner-to-learner interaction builds on the broad educational theories exposed by Vygotsky (1978), which describe the process of learning as a social process whereby the best learning occurs when learners experience a process of interaction and collaboration with other learners and teachers. Commonly used techniques for the promotion of learner-to-learner interaction in online learning environments are grouped as computer-mediated conferencing (CMC) techniques which can be either asynchronous (occurring over a period of time) or synchronous (such as live chat). They enable increased flexibility for students to engage in learning activities in their own time and own place, removing the boundaries of the traditional face-to-face experience. It is thought that asynchronous conferencing gives participants an ‘opportunity to reflect upon each message, providing a considered response’ and the ability ‘to participate in a manner that is more considered and reflective than is normally possible in face-to-face sessions’ (Kirkwood & Price, 2005, p.269).
1

Despite these apparent theoretical advantages in the use of these techniques, there is still considerable research showing that in practice there are barriers to the achievement of these results. While it is clear that students use these mediums for sharing knowledge, providing feedback, and explaining and elaborating on each other’s ideas, researchers have found that students did not regularly negotiate meanings related to concepts, apply their knowledge to new areas and engage in reflective thinking (Maor, 2003, Paulus, 2005). Claims are still made that learning will not be enhanced by these developments until students ‘understand not only how to work with ICTs, but why it can be of benefit to do so’ (Kirkwood & Price, 2005, p.260). This is reinforced by suggestions that students need to learn strategies, be provided with scaffolding and practise reflective thinking processes (Maor, p.131). This paper will extend the research that has been conducted over the past five years using asynchronous discussions in an undergraduate health science degree. Of particular focus in this report is an analysis of the student perspective whereby students in 2004 and 2005 were asked to reflect on their experiences and achievements based on Salmon’s five-stage model of online learning and teaching (Salmon, 2000). Qualitative analysis is used to consider student reflections both before and after participating in asynchronous discussions, along with quantitative analysis to relate this information to the students’ learning outcomes.
The context of this study
At the University of Sydney, the School of Applied Vision Sciences third-year undergraduate orthoptic students are off-campus for a significant part of the semester undertaking professional practice experiences at eye clinics in hospitals or in private sponsored practices around Australia. They are, therefore, unable to offer each other face-to-face support. The asynchronous discussion tool in the learning management system WebCT has been used for the past five years as the central tool to facilitate development of student communities of inquiry. Clinical case-based scenarios addressing both the key content areas and dilemmas in clinical decision making are debated in private asynchronous discussion groups of up to ten students. The first four cases are provided by the lecturer. Students interact with each other by sharing their ideas and conceptions about the different approaches to solving these clinical problems and formulate a group consensus solution to each case which is posted after a period of approximately two weeks. The lecturer then provides feedback to each group individually and also provides a set of model answers addressing the key issues for each case. Later in the semester, students present an individual case in a similar manner and are responsible for moderating the ensuing discussion and providing feedback to the other students.
Orientation activities designed to improve the quality of the student experience
Students attend an orientation tutorial which gives an overview of the complete WebCT site. Research undertaken by the lecturer during the past four years has been used to enhance the orientation sessions to include greater emphasis on the building of a collaborative online group. Orientation activities not only address the technical aspects of asynchronous discussions, but also provide students with tools that enable them to interact effectively, and promote knowledge construction between discussion group members. Content analysis of student postings, using Salmon’s (2000) conference analysis categories, before the introduction of orientation sessions with these activities in 2003 and of postings in 2004, after the orientation sessions, indicated a statistically significant increase in postings that demonstrated interactive rather than individual thinking (Wozniak & Silveria, 2004). That is, students demonstrated more postings where they critiqued other students’ ideas, challenged opinions with further
2

questions, negotiated new meanings and summarised contributions. This fosters the development of higher-order learning and enables linkages to be made between the practical and theoretical components of the course. It was also found that the level of engagement with online discussion significantly correlated with the students’ marks in some of the assessment tasks, particularly the practical examinations where students are required to apply content materials to the real-life clinical cases. Students who achieved higher levels of interactivity with Salmon’s conference analysis categories also achieved higher marks for their clinical assessments (Silveria, Wozniak & Heard, 2004). Students have commented on the success of this process stating that ‘it is such a great way to find out what others think’ and how ‘easily we can freely express our own opinions’. Over the past three years, the average number of postings per student has increased six fold from five to thirty per student, and in 2004 over 1200 messages were posted by thirty-five students.
Stage 1 Gaining Access & Getting Motivated
You can enter the site You can post your first message
Stage 2 Online socialization
Getting to know your group Beginning to share your ideas with others
Interact with others Ask questions about information Explain your ideas, support ideas of others Add examples to your ideas Re-evaluate your opinions Aware of being a “lurker” and the consequences of not actively participating in the group
Stage 3 Information exchange
Stage 4 Knowledge Construction
Use the ideas of others to expand your own Negotiate and interpret ideas and meanings Summarise previous contributions Formulate actions from the shared ideas
Stage 5: Development
Able to critically evaluate your own learning Able to support new comers
Figure 1 Salmon’s model of teaching and learning online with computer-mediated communication adapted for undergraduate students (Salmon, 2000, p.26)
3

Using Salmon’s model to promote reflection about the student experience
In 2004 and 2005, students were introduced to Salmon’s five-stage model of teaching and learning online through computer-mediated communication (2000, see figure 1, above). The model assists the student to progress through the steps of accessing the CMC tools, online socialization, exchanging information, conferencing to construct knowledge and, finally, critical thinking where the learner adopts responsibility for their own learning. Much of the work of Salmon (2000, 2002a) describes the processes that e-moderators can use to assist students to progress through each of the stages of the model. In the context of our resource-poor environment, where there was no emoderator support to facilitate this process, this model was adapted for use by students, with the aim being to empower the students to follow their own development through the five-stage model. The orientation activities were adjusted in 2004 to closely model the five-stage model of learning online and formed the background pedagogy for the use of the discussion tools. Since all moderation was conducted by the lecturer with no additional support from other online tutors, there was a strong emphasis on equipping the students with the skills needed for effective online communication. Other researchers have adopted this model in the design of their ‘e-tivities’ (Pavey & Garland, 2004), but have concentrated their analysis on reflections of the staff experiences and anecdotal student comments rather than analysing the students’ self assessment of their stage of online learning. Pallof and Pratt state that an important element of the online learning experience is the learning that is ‘based on reflection and on the interpretation of experience’ known as transformative learning (1999, p.129). They consider that it is important to create a space to ‘open the door to reflection’ (p.135) and suggest that when students are asked to evaluate their participation, they include a self-reflective exercise. By introducing Salmon’s model of learning online to students, and supporting its use with additional self evaluation and reflection activities described in the next section, it was hoped that this would provide a framework for fostering transformative learning experiences.
Analysis of the student experience
In 2004 and 2005, a total of forty-eight students were surveyed before and after participating in online discussions. At the conclusion of the orientation activities in the second week of the semester, they were asked to consider their early experiences and provide comments based on the areas outlined in table 1 (below). They revisited this survey information at the completion of the semester and were asked to compare their responses at the beginning of the semester with their actual experiences. They were asked particularly to use a process of reflection to gain insights into their experiences by considering what this might mean for their future learning experiences. The numbers reported in the results vary as not all students completed both surveys. Ethics approval was gained to enable us to gather this data and compare it to each student’s assessment results. This consisted of their online participation assessment which was determined by the quality and quantity of their postings. The overwhelming majority of students reported that they accessed the online materials at home (67%) or at home and university combined (30%), which was consistent with their expectations at the commencement of semester. It appears that difficulties related to equity of access experienced in our early research in 2000, when online materials were first introduced, have been largely overcome. This is similar to overseas experience (Kirkwood & Price, 2005), though Lobry de Bruyn (2004) notes that distance students in rural areas of Australia may still have accessibility issues. Students reported that they accessed the asynchronous discussion between two and five times per week, which did have a tendency to reduce towards the end of the semester when competing workload demands were higher. As expected, the students’ self assessment of their stage of
4

development as online learners using Salmon’s model showed a significant change from the beginning of semester to the end of semester as shown in figure 2 (below) (Wilcoxon Signed Rank Test, -4.076 p<0.001, n=30). However, the pattern of change across the group was not consistent and predictable (Pearson’s correlation, 0.335, p=0.71, n=47). The mean stage from Salmon’s model reportedly achieved by students in week 2 of the semester was 2.4, n=32 (between stage 2 online socialization and stage 3 information exchange) and in week 14 at the end of semester was 3.7, n=47 (between stage 3 information exchange and stage 4 knowledge construction). Table 1 Data collected before and after participating in asynchronous discussions in 2004 and 2005
Area 1 Information sought at commencement of semester (BEFORE) How they anticipated they would access the online communication tools (home/ university/ other locations) and how often (per week) Comments regarding their prior experience with online learning both the positive and negative aspects What they hoped they would be able to achieve from participating in online discussion groups What stage they believed they were currently at on Salmon’s model and why Information sought at the conclusion of the semester (AFTER) Differences in access and number of accesses
2
Comments about the actual online discussion experiences – positive and negative aspects Achievements they felt they had made through participation Stage they were currently at from Salmon’s model and why Additional comments or suggestions Assessment results – online participation marks
3
4 5 6
50 45 40 35 30
% 25
20 15 10 5 0 1 2 3 Stage 4 5 Week 2 Week 14
Figure 2: Student self assessment of stage from Salmon's model
5

Table 2 Grading scale for reflective comments (adapted from Hatton & Smith, 1995)
Type of Reflection Description Student reports events. No attempt to provide reasons or justification for events. Not only a description, students make some effort to analyse reasons for events or actions from their own point of view. Example I’d read most of the postings, sometimes print them out, and read whenever I had time. I also tried hard to contribute to the group discussion, by getting references and writing up information in detail. I really enjoyed logging on and reading other people’s opinions, as often they were slightly different to my own, and brought up issues I did not think of. I feel that I contributed well to my group, and I am not sure that I would change anything next time around. The only thing that I may do next time is rather than relying on someone else in the group to provide an answer to an issue that has been raised, I should really research it myself. … Having said this, however, I also feel it is important when working as a group that all the group members feel valued and feel like they can contribute somehow. I know that in future I would be more confident, experienced and aware of the online culture and so there would be less of a developing stage, more of a revision stage when involved in e-learning. This would allow my contributions to be more effective. I also only became aware after some time that I should make a comment to show that I have read the discussion and am aware of the conversation, even if I did not contribute. In future I will know to do this from the start. It has provided me with another way to learn. First in a group and second by making me think and give my opinion of what I believe to be the answers. This is a good way of learning because in reality, in our profession, everyone can have a different opinion in different cases. Through these online discussion groups we as a group have been able to discuss a huge variety of cases that we could be faced with in the future and therefore as orthoptists we will have to make informed decisions based on those kinds of results to come up with a diagnosis and hence management. Online learning has allowed me to not only read what others think, but also to question why they think that and then come up with a good conclusion that everyone agrees on. It has allowed me to take on a different approach when looking at a situation and how to analyse it.
1
Unreflective descriptive writing Descriptive reflection
2
3
Dialogic reflection
More complex as students ‘step back’ from events/actions leading to a different level of mulling about, discourse with self and exploring the experience.
4
Critical mode
Shows greater insight by considering multiple contexts, such as their view in the light of others and how it affects others in the group process. Has awareness of the role that the experience plays in learning and how activities shape their learning or future practice.
Qualitative data (areas 2, 3 & 5 from table 1) were analysed independently by two researchers to identify common themes and also coded separately according to four categories describing the depth of the reflective comments. This grading was based on a framework developed by Hatton & Smith (1995), which describes different levels of reflective writing and is detailed in table 2 (above)
6

with examples from this study. Areas of disagreement between the two researchers were discussed and resolved. Students reported overwhelmingly that their experience of asynchronous discussion was positive (89%), with a small number stating that the experience had both positive and negative aspects (8%). Comments related to the many advantages that a flexible learning environment affords learners, such as: ‘I learnt a lot from my peers in my own time’, ‘it allows you to judge your own standards against others’, ‘continual feedback that you get is a good indicator of your progress’, ‘encouraged deeper learning’. Some students did comment on difficulties in keeping motivated across the whole semester, feelings of frustration when group work was not shared equally, and the fact that they would rather discuss issues face to face because they had difficulties expressing their opinions in writing. Despite these comments, the majority of students appreciated the collaborative nature of the online learning environment and also the additional support and feedback provided to enhance their learning experiences. Figure 3 (below) outlines the results of grading the reflective comments. Once again, there was a significant change in the level of reflection over the semester with a general improvement in the depth of the reflections (Wilcoxon Signed Rank Test, -4.243 p<0.001, n=36). The pattern of change over the semester was predictable, with students generally moving to more reflective comments by similar amounts (Pearson’s correlation, 0.551, p=0.000, n=36).
50 45 40 35 30
% 25
20 15 10 5 0 1 2 3 Quality of reflective statements 4
Week 2 Week 14
Figure 3: Grading of Reflective statements
When considering the relationships within the data it was interesting to note that there was a significant correlation between the students’ self-reported assessment of their stage of online learning with Salmon’s model and the grading of their reflective comments at the end of the semester (Pearson’s correlation, 0.328, p=0.026, n=46). There was also a significant correlation between the stage on Salmon’s model at the end of the semester and the online participation marks (Pearson’s correlation, 0.411, p=0.005, n=46). The related correlation between the grading of reflections and online participation was also significant but not as strong (Pearson’s correlation, 0.335, p=0.021, n=46).
7

Implications for the improvement of the student experience in online discussions
Until now there has been a focus on the strategies that e-moderators should use to ensure that students participate and engage in online discussion activities. This has included considerable research regarding the types of activities and tasks that promote student engagement and interaction (Paulus, 2005), and the role of the e-moderator to ensure student interaction (Spector, 2005), often using content analysis of the postings made in discussion groups or qualitative student comments about their experience. All are used to derive useful strategies to enhance the quality of the student experience in asynchronous discussions. Many conclude that students need to be taught how to interact effectively in the online learning environment. The gap in asynchronous discussion research has been the use of a structure for examining the students’ perspective regarding their development as online learners. This research has provided a framework for analysis of the student experience by empowering the student to track their own progress as an online learner after carefully planned orientation activities. Results from this research and that undertaken previously (Wozniak & Silveira, 2004 and Silveira, Wozniak & Heard, 2004) have shown that undergraduate students are able to progress to stage 3 and 4 of Salmon’s model, and have positive experiences with this process which correlates with their learning outcomes. Wu & Hiltz (2004) used a questionnaire to examine students’ perceptions of their learning after using online discussions. They found that students who were more motivated and enjoyed their experiences also reported higher perceptions of their learning achievements, although they recognised that there may be a difference between perceived and actual learning. Webb et al. (2004) have also noted a positive association between participation in what is termed ‘e-learning dialogues’ and learning outcomes. The number of student postings and number of times each student accessed the discussion board were correlated with the student’s participation mark. They do note, however, that both active (posting responses) and passive (reading contributions of others) participation was related to learning outcomes achieved, suggesting that ‘lurking’ in discussion forums can also have a positive impact on learning. Content analysis of online discussion postings was carried out by Mehanna (2004) and clusters of pedagogies were identified and related to student grades in a postgraduate program. It was found that if students engaged in online discussion activities, such as providing feedback, summarising and notetaking, reinforcing effort, building arguments and engaging in cooperative learning were positively correlated with student grades. All of these activities are consistent with the types of activities that are highlighted in the adaptation of Salmon’s model described in figure 1 of this paper. It needs to be recognized that there are many factors that may influence the effective use of asynchronous discussions and learning outcomes achieved and it is difficult to isolate these in this research. It is suggested by this research that Salmon’s model combined with adequate preparation of students can provide a useful framework of the online learning experience. This is largely at odds with the findings of many others who suggest that considerable e-moderator support is required to monitor and maintain a collaborative learning environment with effective student interaction. An area that could benefit from further enhancement is the role that reflection plays in the development of more effective online learners. Salmon (2002b) suggests that online reflection is akin to providing a ‘mirror on the screen’ and is important in the design of an online learning experience. She discusses how reflection can benefit both the individual learner and group learning, but suggests that methods of encouraging critical reflection require further investigation. It is apparent from other researchers in traditional (Hatton & Smith, 1995) and online learning environments (Maor, 2003) that students struggle to achieve these higher levels of reflective practice (described in table 2). Methods of scaffolding reflection activities have been suggested (Ward & McCotter, 2004; Rose & Devonshire, 2004), including the use of a guided rubric
8

introduction and a staged process to assist the student to move to higher levels of reflective practice.
Conclusion
Online learning environments increasingly are being utilised in both blended and distance education programs with asynchronous discussions seen as a means of promoting collaboration and enhancing the quality of the student learning experience. When students are orientated to the learning opportunities available to them in online environments and given a structure to assist them in monitoring and evaluating their experiences, positive learning outcomes can be achieved. This is possible in the increasingly resource-poor environment of higher education in Australia today without requiring ongoing e-moderator support and the ensuing time drain associated with assisting students in this environment.
Copyright ? 2005 Wozniak, H. The author assigns to ODLAA and educational non-profit institutions a nonexclusive license to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author also grants to ODLAA a nonexclusive license to publish this document in electronic or print form within ODLAA publications and/or the world wide web. Any other usage is prohibited without the express permission of the author.
References
Anderson, T., Annand, D. & Wark, N. (2005). The search for learning community in learner-paced distance education: or ‘Having your cake and eating it to!’ Australasian Journal of Educational Technology, 21 (2), 222-241. Hatton. N. & Smith, D. (1995). Reflection in teacher education: Towards definition and implementation. Teaching & Teacher Education, 11 (1): 33-49. Kirkwood, A. & Price, L. (2005). Learners and learning in the twenty-first century: What do we know about students’ attitudes towards and experiences of information and communication technologies that will help us design courses? Studies in Higher Education, 30 (3), 257-274. Lobry de Bruyn, L. (2004). Monitoring online communication: Can the development of convergence and social presence indicate an interactive learning environment? Distance Education, 25 (1), 68-81. Maor, D. (2003). The teacher’s role in developing interaction and reflection in an online learning community, Education Media International, 40 (1/2), 127-137. Moore, M. (1989). Three types of interaction, American Journal of Distance Education, 3 (2), 1-6. Mehanna, W.N. (2004). e-Pedagogy: The pedagogies of e-learning, ALT-J, Research in Learning Technology, 12 (3), 279-293. Pallof, R.M. & Pratt, K. (1999). Building learning communities in cyberspace: Effective strategies for the online classroom, San Francisco: Jossey-Bass. Paulus, T. (2005). Collaborative and cooperative approaches to online group work: The impact of task type, Distance Education, 26 (1), 111-125. Pavey, J. & Garland, S. (2004). The integration and implementation of a range of ‘e-tivities’ to enhance students’ interaction and learning, Innovations in Education and Teaching International, 41 (5) 305-315.
9

Rose, M. & Devonshire, E. (2004). Rethinking reflection: Using online reflective learning in professional practice for Indigenous health workers, Education Media International, 41 (4), 307313. Salmon, G. (2000). E-moderating: The key to teaching and learning online, London: Kogan Page. Salmon, G. (2002a). E-tivities: The key to active online learning, London: Kogan Page. Salmon, G. (2002b). Mirror, mirror on my screen … Exploring online reflections, British Journal of Educational Technology, 33 (4), 379-391. Silveira, S., Wozniak, H. & Heard, R. (2004). Using online discussion groups to enhance the learning outcomes of undergraduate orthoptic students. What have we learnt? In D. Verlohr, A. Rydberg & Z. Georgievski (Eds.), Global perspectives converge down under: Transactions of the 10th international orthoptic congress, (359-362). Melbourne 14-17 November 2004. Spector, J.M. (2005). Time demands in online instruction, Distance Education, 26 (1), 5-27. Vygotsky, L.S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. Ward, J. & McCotter, S.S. (2004). Reflection as a visible outcome for preservice teachers, Teaching and Teacher Education, 20, 243-257. Webb, E., Jones, A., Barker, P. & van Schaik, P. (2004). Using e-learning dialogues in higher education, Innovations in Education and Teaching International, 41 (1), 94-103. Wozniak, H. & Silveira, S. (2004). Online discussions: Promoting effective student-to-student interaction. In R. Atkinson, C. McBeath, D. Jonas-Dwyer & R. Phillips (Eds.), Beyond the comfort zone: Proceedings of the 21st ASCILITE conference, (956-960). Perth, 5-8 December. Retrieved June 21 2004 from https://www.wendangku.net/doc/ec14820979.html,.au/conferences/perth04/procs/wozniak.html Wu, D. & Hiltz, S.R. (2004). Predicting learning form asynchronous online discussions. Journal of Asynchronous Learning Networks, 8 (2), 139-152.
Acknowledgements
I would like to acknowledge the assistance provided by Sue Silveira in the analysis of the qualitative data and Dr Rob Heard for the statistical analysis.
10