Engagement and motivation through badges

Tamagotchi by stopsign, on Flickr
Creative Commons Attribution-Noncommercial-Share Alike 2.0 Generic License  by  stopsign 

Feed me!

When considering what motivates students to participate in a learning intervention, learning designers spend a lot of time designing activities to engage and interest students. However, research indicates that students tend to respond to the assessment regime rather than the learning objectives. This means that students will look at what is required to pass the course (the hidden curriculum) and then make sure that their activities focus on what is required to fulfil the requirements of that particular task (Gibbs, 2010).

I have found this when working in a corporate e-learning environment, where the learners’ objective is to get through the multiple choice quiz as to be deemed competent, so the approach is to quickly click through screens of an elearning module (or in some cases ask a colleague to do it). In my Master’s course some students have never appeared in any online discussion forum or participatory activity as it is possible to pass the module by turning in the written assignment (which I presume they did). This is understandable and a sensible strategic approach to gaining a qualification or fulfilling the needs of a workplace learning scheme. Is it learning? Well, it’s really impossible to say what the participant has or hasn’t learned but they have not ostensibly participated in the learning process that has been designed by the instructional designer or faculty.By process, I mean the pathway comprising the range of activities that are designed to help the learning and practice of skills and competencies that will help the learning objectives – in short the learning design. For example, an activity to  stimulate dialogue and build peer review skills might ask students to read a paper on a subject and post a summary and response in a blog post, inviting comments.

Non-participation in activities become more visible in an online or blended environment as students dn (or do not not) leave digital footprints. Yet,  as these activities are ‘just learning’ and have no bearing on the assessment (typically an end of module exam or paper), then a significant proportion of students don’t bother. This can diminish the effectiveness of online courses which often are designed for and depend on learner-learner interaction.

If it is assessment rather than the learning activities that motivate (or spur) the larger proportion of students to participate in the learning process, a close alignment of the learning and assessment process could motivate more students to participate. This is in itself not a new concept: good teachers have always taught in a way that aligns the learning that takes place in a classroom with the assessment. This could mean that artefacts that are produced as a result of learning activities are included as part of or necessary to the assessment itself, or that assessment is ongoing requiring frequent participation and engagement. In formal online learning and in MOOCs this poses a challenge due to the distributed and generally asynchronous nature of online learning, and is used with care by learning designers as it then affects the flexibility aspect of online learning. After all it is this type of learning that attracts  the type of person who might not be able to study in any other way (formal campus based learning requiring the necessity of being in one place at a certain time). Working professionals or people with caregiver responsibilities may appreciate the value to flexibility that online learning enables.

What methods or strategies might spur engagement or at least encourage learners to participate in the learning process as the learning designers intended? I wrote about Open Badges for my final (possibly ever) assignment for the Open University Masters in Online and Distance Education. Open Badges are digital artefacts that learning designers can use in a course to recognise achievements or reward participation at a lower level of granularity than traditional assessment and so might be able to motivate learners due to the element of immediate gratification or gaming.  Learners can earn badges for specific tasks within a course that acknowledge a skill or completed activity, thus building and visibly showing progress and gaining something tangible they can show on a profile. Open Badges are embedded with meta data and can be displayed by the earner in a medium such as a personal blog or a LinkedIn profile. A third-party reviewer of the badge can click on it to learn more about the skills attained and even access the original piece of work that was required to earn the badge.

Badges therefore give learning designers another assessment-type tool, less scary then a formal assessment, and collecting a number of badges can be rolled up into a larger super -badge which could provide a pathway to another form of certification. If a course designer chooses to use badges for motivation and engagement, they need to decide what activities are worthy of a badge. This in itself gives the learner an indication of the importance of the activity and/or the type of skills that will be developed if the learner chooses to earn the badge. Badges can help make the learning design explicit, perhaps helping to bridge the intention of the learning designer and the understanding of what is important for the learner.

Earning badges therefore gives learners opportunities to collect rewards, but this still might not appeal to learners who feel they have nothing really to gain. Using the idea behind loss aversion,  what if users were given a number of badges at the start of a course – partially complete perhaps. If they participate in  learning activities or meet certain targets, they  gain more badges. However, if they do not participate, they  lose badges. I wonder whether the prospect of losing badges might be more of motivator than gaining badges (you can’t lose what you haven’t got yet). This sounds one of those digital pets you have to keep feeding to keep alive :-). What if in order to get through the course, you have to metaphorically keep feeding it? How you do this might have elements of choice, but the opportunities inherent in badges such as expiry dates or the ability to hold more or less information poses some interesting learning design opportunities.

References

Gibbs, G. (2010) Using Assessment to Support Student Learning, Leeds, Leeds Met Press.

Mozilla, (2012) Open Badges for Lifelong Learning [Online]. Available at https://wiki.mozilla.org/File:OpenBadges-Working-Paper_012312.pdf

Storyboards and prototypes

This post reflects on constructing a storyboard  and a prototype for a mobile learning outdoors activity as part of the Learning Design Studio for H817 Openness and Innovation in elearning

Imagining – the storyboard
The storyboard is a visual mapping out of the proposed learning activity. Although we were working in a group, we created individual storyboards first. My storyboard grew out of the work done at the research phase, and I also used the personas’ forces to inform the design of activities. I created mine in Linolit, which worked well for the creation but was not easy to output into a format that could be embedded in a website or document, so in the end I took a photo of the screen.

Sukaina_storyboard_pic copy

Storyboard (click for a larger image)
My storyboard spanned three phases and detailed a number of activities. While I was developing this, it became clear that I was designing for two audiences: school groups and day visitors but that the school trips would need activities that spanned beyond the site visit. I therefore developed an approach that had classroom activities either side of the site visit, but that the activities at the site visit could also be done stand-alone using resources available either at the Visitor Centre or through the website. During this time, it was the case studies that I had analysed that provided the greatest influence although I also referred to the personas when thinking would this persona be interested in this activity?

Two team members completed storyboards and three members discussed the storyboards in Google Hangouts and agreed to synthesise the two boards using the Google template provided as there were considerable similarities, adopting the overall approach of a learning activity that spanned classroom and on-site learning. Following this, one team member agreed to construct a team storyboard which would be approved by the whole team in another collaborative meeting. This stage also saw a narrowing of the scope to just myths and legends, rather than a more general all-encompassing approach that might have covered other subjects. The team discussion was a strength here as the team members were able to use the visual aspects of the storyboard to discuss options and differences. The advantage of the collaborative storyboard was its visual display of the learning activities; we could see which activities were cluttered and unclear, and we could move them around, to see what we were expecting participants to do. I could see that the process was important for helping the activity form in our imaginations; for example we placed activities for a given stage in a non-linear way, which intuitively showed that the activities did not need to follow a prescriptive stage and were more fluid.

The storyboard was not without its problems though as in order to fit text into boxes and even into the overall canvas, we may have oversimplified some things and the level of granularity needed for description was not always clear. The storyboard worked at an activity level, but I am not sure how it would work at a course level with the level of collaboration and discussion that was required.

Making it real – creating a prototype

Before we could build a prototype, the team had to decide what features to prototype. The mechanism for this was in developing a features table from the storyboard, extracting features from the storyboard. This provided to be a useful activity, not only for deciding which features were important to prototype but also served as a reality check of whether the activities on the storyboard made sense. The features table comprised a list of items broken down into discrete components such as creating the text instructions for a page, the actual page (say a web page), any media elements associated with that page, and any functions that might need to be built into a web page. There was initial confusion about the task  as it required some understanding of what was meant by a ‘feature’ and also re-translating the original instruction of organising by ‘scene’ (as in a movie or series of screens?) to something that made sense to our project. For this, I split our activity into 4 scenes: scene 1 was activity in the classroom prior to the site visit, scene 2a was activities at the Visitors’ Centre, scene 3 was Activities in the outdoors at the Giant’s Causeway and scene 4 was post-visit activities to create the artifact. Once this organising principle was in place, it became easier to extract the features.

Creating a prototype for an outdoors activity for a mobile learning app was a challenge as we did not have a realistic chance of actually building an app, and we were working in a virtually distributed team. However, putting myself in the learners’ position helps prompted me to suggest that we could develop the website that would serve as the ‘home’ for the activity – where users could go and download the app and the brochure, where users could go to see what stories had been told, where teachers could go and find more information. This at least would serve as an authentic experience for someone deciding whether to partake in the activity, planning the field trip or who wanted to submit their story.

Further in order to prototype the app functionality, we agreed to create PowerPoint presentations of the steps the user would go through to see the augmented reality and the other features about the app.  There was no time to actually develop the app but we were able to create one example of augmented reality which was recorded and uploaded onto the site.

I quickly created this site using Google sites adding holding text into the relevant sections, and then another team member made the powerpoint presentations that demonstrated the some of the screens of the app, showing what functions were available.

prototype

Screenshot from prototype website showing mobile app interface mockup

Constructing the prototype and seeing parts of the possible product come to life was a motivating moment as there was something tangible beyond the discussions, ideas and the storyboard.

Prototypes such as this have their limitations. In this case, the heart of the activity is the mobile app, but without the time or resources to build the app, we could only demonstrate in a presentation how some screens and features of the app might work. This is a limited way to assess whether a mobile learning activity is going to function in the context, where connectivity, speed as well as other environmental variables might be a factor. These aspects might not be picked up in a prototype, and I’d argue for a mobile app, that a beta version would be required for adequate testing in the field.

However, the prototype we built was useful to communicate what the activity might involve for learners and field trip organisers and would be useful for a situation to establish a proof of concept or to get buy-in.  Simple mockups can prevent expensive programming that has to be discarded or (possibly worse) continuing with a problematic product because it is too late or expensive to make changes.

Researching the design challenge

This post reflects on the research conducted at the Inspiration and Ideation stage of the Learning Design Studio, which is part of the OU module H817 Openness and Innovation in elearning.

The Inspiration and Ideation stage of Learning Design was to review case studies and theoretical frameworks to develop design patterns and principles. The purpose of this was address the design challenge based on evidence of what might have been done before and from case studies of learning designs in similar contexts.

The challenge

In my previous blog post, I described the process of articulating the context. The context refers to the design space and the concerns of the learners and is what the designers have to work with. The challenge is the change that is desirable to be effected in that context. For our project, the challenge was: 

to design a mobile learning application that will facilitate learning through the exploration of the UNESCO world heritage site the Giant’s Causeway, Northern Ireland. The project is geared towards young people from the ages of 10-14, but will also be applicable to overseas tourists and students. The challenge is to engage the learners while at the site as part of a broader seamless learning approach, where the learning can be consolidated when back in the classroom. Typically, the learners will visit the site as part of a field trip, and the challenge is to use mobile and social devices to support the learning goals around understanding and appreciating the historiography of the site including discussion around legend, myths and how people in history have interpreted the site and how it came into being.

It was important to articulate the challenge so that the type of research conducted would look for a similar contexts such as field trips, local history, learning in outdoors environments (as opposed to museums) and use of mobile devices.

Reviewing the evidence

I chose two case studies which seemed to align with the context and the challenge. This was an enjoyable and interesting  activity that increased my own understanding of mobile learning ‘in the wild’, approaches to learning about history using augmented reality and geo-location, and group interactions in informal learning contexts with mobile devices.

  • The first case study was a comparison of offering audio tours around Nottingham, UK (Fitzgerald et al., 2013) which compared a people-led tour (with human guides narrating scripted audio) and a technology-led tour (with smartphones where GPS activated audio at selected points) on the subject of the issues around the interpretation of the 1831 Reform riot.
  • The second was an account of designing field trip activities for primary level students visiting the Chinatown area of Singapore using mobile devices (So et al., 2009), where the learning outcomes were to facilitate knowledge building in an outdoor location. This study demonstrated how the design evolved from an instructivist approach of consuming information at points in the field trip to one where students used Google Maps to track their own journey, making and sharing  notes and observations with peers and culminating in a review of the artefacts students made.

For each case study, I wrote a design narrative summarising the key points and lessons that might be derived for designing mobile learning in the field. Another team member contributed case studies and narrative, so the team had a body of evidence from which to derive design patterns and principles.

The Learning Design process instructed the team to write design patterns and principles derived from the insights gained from analysing the case studies in order to ‘formulate [the] insights into “building blocks” for design’ (OU module materials, Mor, 2013).

Despite the existence of a template, deriving the patterns was challenging partly because I wasn’t sure what level of granularity or generality to aim for in the patterns. Using the design narratives as the source, I devised a pattern about promoting critical thinking and higher order skills when on a field trip through clear learning goals (preferably learner derived) as well as specific activities.

In addition to the case studies, I also chose a Theoretical Framework for Mobile Learning (Herrington et al, 2009), which informed writing a principle of using the learners’ own devices and one about using mobile devices to produce as well as consume knowledge. These were uploaded to the group’s project site for review and discussion.

The case studies, design narratives, patterns and principles helped to develop some key design directions which the team could agree on. The possible tensions between mobile devices as distractions in an outdoors setting and the possible negative effects of group cohesion was something we had already identified in the personas’s factors and concerns and which surfaced in the patterns and principles. so we knew we’d need to take account of this in the design. The opportunity of using the affordances of mobile devices for enhancing learning in contextualised situations and at outdoors sites was an exciting prospect to inform the design, especially where co-creation of content and development of an artefact could be part of the activity.   Thus, the process also informed what we would not do, in this case only deliver media to users on mobiles at points at the site, and we would not make the activity linear and prescriptive. These insights helped when it came to developing the storyboard.

Reflections

Reviewing case studies of similar contexts and challenges and finding theoretical frameworks that can inform a design is something that I’d do instinctively, especially if it is a context I am unfamiliar with. This activity took a more structured approach to reviewing  case studies and theoretical frameworks with the additional instruction to derive patterns and principles. The idea is to have a bank of patterns and principles that a learning designer can call upon. Writing a pattern was unfamiliar  and it took some time to conceptually understand not only the purpose but also what a pattern looked like. Was it a recipe? Was it a FAQs? Was it a do’s and don’ts? What is the level of detail? Who is the pattern for? How makes a good pattern? Are patterns peer reviewed?

Looking at some other patterns was useful, but many of the examples we were pointed to seemed incomplete. This is one of the activities I would like to go back to, understand better and consolidate, as although I derived a number of patterns and principles for our project, I am not sure that in a future real-world project,  I would go out and seek other patterns (where would I go? Wouldn’t it be better to read the actual case study or design narrative?), nor would pattern writing be something I would necessarily include in a learning design process I was managing, where time becomes a constraining factor.

One final thought about the importance of evidence-based design is that it brings a level of objectivity to the process.  While this was not the case in our team, developing narratives, patterns and principles could also help diffuse tensions and disagreements in teams or with stakeholders holding differing views about a design direction, by using the objective (but still interpreted) evidence to inform design.

References

FitzGerald, Elizabeth; Taylor, Claire and Craven, Michael (2013). To the Castle! A comparison of two audio guides to enable public discovery of historical events. Personal and Ubiquitous Computing, 17(4) pp. 749–760. Available at: http://oro.open.ac.uk/35077/

Herrington, A., Herrington, J. and Mantei, J. (2009) ‘Design principles for mobile learning’ in Herrington, J., Herrington, A., Mantei, J., Olney, I. and Ferry, B. (eds) New Technologies, New Pedagogies: Mobile Learning in Higher Education, Sydney, Faculty of Education, University of Wollongong, pp. 129–38.  Available  at http://ro.uow.edu.au/ edupapers/ 88/

So, H.J., Seow, P. & Looi, C.K. (2009). Location Matters: Leveraging Knowledge Building with Mobile Devices and Web 2.0 Technology. Interactive Learning Environments, 17(4), 367-382. Available at: http://www.editlib.org/p/64762/

Using personas in learning design

This post reflects on using personas in learning design and in particular how I worked with personas during Block 3 of H817 The Learning Design Studio.

In a previous post I made some general comments about the Learning Design Studio process. In this post, I’m looking back on the key aspect of defining the context and in particular understanding who to design for.

In some of the work I do with designing or developing learning experiences, the learners can often be seen as an amorphous group of people ‘somewhere out there’. Conversely, learners are seen as a highly disparate group who have few common characteristics (they may be mixed ages, genders, educational level) and so attempting to design for this group is seen as either very difficult or just impossible. Another approach is to attempt a universal design approach for learning that caters for all, but this approach is not without its detractors. It’s arguable whether it is possible to achieve universal design beyond a rather general and technically focussed approach.

So it’s been revealing working in Block 3 of H817 Learning Design Studio, where a key part of the learning design process has been to engage with personas. Personas are defined as

‘ ‘Fictional characters created to represent the different user types within a targeted demographic, attitude and/or behavior set that might use a site, brand or product in a similar way.’(Wikipedia , 2012) ‘

The advantage of developing personas is that in thinking of users as real people with names, specific details and needs, they cease to become ‘elastic’, and it is the product or learning design that needs to bend to them rather than the other way round.

Developing personas for the Exploring Giant’s Causeway project

In our project, three team members each developed two personas based on the vague outcome that the site would support visitors to the Giant’s Causeway site. Initially, I came up with the following personas:  Marcia, a Texan-based university student, Yuan – a British educated Malaysian father with two daughters visiting the site and our most memorable persona ‘Ryan’ – a sightly disgruntled assistant manager in the Giant’s Causeway Visitor Centre. My colleagues developed equally engaging personas, and interestingly both of them independently developed American personas with Irish roots.

ryan_persona_blog

Snapshot of a developed persona in the persona template

Working with personas helped to make the design real. Initially the process seemed contrived as I felt I was writing the cast of a potential soap opera. The task was to use the persona templates to create personas – detailed identities with names, pictures and personal habits of individuals for whom the learning was to be designed or who might be a stakeholder in the process.

Yet after this initial wave of (somewhat enjoyable) persona writing, I was struck with richness of the discussion that followed amongst the group and saw practically how the process of developing personas helped to define the context. We realised that the type of learning ‘in the wild’ we might want to develop around the myths and legends of the site might be more suitable to school children, family groups and teachers rather than tourists. Details in the personas’ profiles such as mobile phone roaming charges for overseas visitors,  the desire of tourists to travel in groups and have fun, as opposed to more directed field trip organisers helped us to develop and narrow the context. We also considered what was already available for tourists at the Giant’s Causeway – a reasonably sophisticated audio tour.

Therefore, although school children, teachers and family groups had not featured strongly in our original group of personas they emerged as the people we would design for during the discussion. So, we went back to the drawing board, we culled some personas, enhanced some we had currently and created two new ones (a school girl and a head teacher). I took inspiration from the Giant’s Causeway Trip Adviser site to develop Ryan and Yuan’s personas further, while a colleague based a new Headteacher persona on a real-life school.

In some ways, our work was still limited due to time but useful. For example, we kept Yuan but enhanced his profile as a father of two daughters. If we’d had more time, we might have considered making up separate personas for his daughters. We developed Ryan as he has to deal with school children and teachers as part of his job, and we still acknowledged the importance of other visitors to the site who could still make use of the product.

This discussion helped keep the activity real, as during the time I was developing personas I had a niggling doubts that I might be missing some key learners  and how could I possibly cover the full range of possible users?

Reflections on using persons

Personas therefore provided a valuable means of discussing some of the realities of the potential learners’ situation. The potential learners became real. We worried about whether all the school children would have a mobile device, even though we knew that some schools would provide some devices. We worried that parents were unsure about letting their children have mobile devices and the cost of data might be prohibitive. We thought about Ryan’s attitude that people coming to the Giant’s Causeway should be more concerned about enjoying the scenery than huddling around a mobile device might preclude him from promotion the use of augmented reality at the site.

Initially, the personas exercise for a learning design seemed rather meaningless as there were potentially so many variables for this type of project – an outdoors learning in the wild using mobile devices. However, that in itself became a useful pointer to deciding that it was necessary to narrow down to a group of somewhat connected individuals. In a way, persona development was akin to putting a stake in the ground and making some assertions about possible learners. In setting the context, the persona’s factors and concerns helped flesh out some of the design challenges. The approach has its limitations where the group may have overlooked a major stakeholder, but working in a group meant that there could be a reasonable degree of confidence that a range of users, their factors and concerns had been covered.

 

 

Learning about Learning Design

This post reflects on Block 3 of H817 Openness and Innovation in eLearning. It focusses on my experience as a student working in an online design studio to design a mobile and social learning application for a local history site.

Block 3 of H817 Openness and Innovation in e-Learning has been immersed in the practice of Learning Design (LD). I’ve been working with a small group of other students in an online ‘Learning Design Studio’ to design an activity or learning interaction that supports learning about local history using mobile or social devices. Our project (chosen by the group) has been to bring the myths and stories about the Giant’s Causeway to life for visitors to the site and to help learners investigate and understand how the physical characteristics of the site led to the myths and legends.

The project itself is not the main purpose of the block; rather, the project serves as a vehicle to expose students to the practice of Learning Design (LD) which differs from (although is related to) instructional design (Brock & Mor, 2012). Adapted from software development, this approach to designing learning views the learning task as a problem to be addressed through design, with a focus on the learners’ contexts and the given situation in which a change is desired. It adopts a more constructivist approach to designing learning and is more about designing for learning rather than designing learning.

The overall approach in terms of process is summarised in the graphic below (courtesy of the Open University).

(Click for a larger picture)

LD outline
As the project is concluding, I’ll be posting a series of reflective blog posts about some of the key stages of the learning design method in preparation for the formal assessment. These include articulating the context and the use of personas, designing through research; prototyping, and evaluation. The rest of this post, however, makes some general observations.

General reflections
Overall, this has been one of the most challenging blocks due to a combination of the intensity of group work and the nature of the subject matter. The Learning Design methodology in detail was new (to me at least) and required students to understand theory and apply practice at the same time. Some areas (yes I’m talking about creating pedagogical patterns) were downright confusing in their instructions and application, while others such as storyboarding were just time-consuming, especially with the need to discuss decisions with the group and collate individual artifacts into group-owned ones.

The pedagogy behind the module has been one of peer collaboration and peer feedback with a certain amount of tutor support while following a fairly rigid template-driven format. However the tutor support has been more of a trouble-shooting nature rather than one of guiding as the More Experienced Other and where at times (apologies to Vygotsky here) the Zone of Proximal Development has seemed somewhat far away. At various points, I could have done with a bit more ‘Sage on the Stage’, if only to be able to ask questions and get clarification on the activity.  Evidence of the chatter on tutor groups, (the secret Facebook group), in Google Hangouts and in the Google Community suggest that many students were bemused and flummoxed with some of the instructions on this module. Indeed the one live Elluminate session set up for students to have a discussion the module chair proved useful.

Despite the initial questions and uncertainty, the process has become clearer and more explicit in its purpose as the weeks have gone by and I have adjusted to the pace, the groups, the nature of the various spaces we have to interact in: the Google Project site, Open Design Studio (a social web environment), The Tutor Group Forums, the Course website as well as the student -set up spaces for collaboration, in our case a Google+ Community. Nevertheless, there were times when I had 20 tabs open in a browser and kept getting lost as to where I was and what I was supposed to be doing where. I have stretched my digital literacy skills and am rapidly developing new ones.

There have been many positives; I’ve been fortunate with other members of my group who have embraced the challenge with a sense of humour as well as dedication and with the subject matter. The project –  to design an learning activity for using mobile and social technology for a local history site – has the added bonus of being able to learn about a historical site and to play with tools such as augmented reality and view gorgeous videos of landscapes and stories. In researching an appropriate design approach, I’ve learned a tremendous amount about mobile learning ‘in the wild’, about the pedagogies that might underpin seamless and ubiquitous learning, the challenges of designing learning in the outdoors and how location-awareness and augmented reality might build just-in-time contexts for learners at a particular moment.

While I don’t think I will miss this block (as I did with Block 2 MOOC), the almost regimented and template-driven process of learning design is not one I will easily forget and I have already begun using some of the ideas in real-world context. And visiting the Giant’s Causeway is now on my list of holiday plans.

References

Craft, Brock and Mor, Yishay (2012). ‘Learning Design: reflections on a snapshot of the current landscape’. Research in Learning Technology, 20. Available at http://www.researchinlearningtechnology.net/index.php/rlt/article/view/19196