Recordings of speakers from the Hybrid symposium event at UCL in June ’22 are included below for convenience and quick reference. The videos below comprise recordings of Ruth Puhr, Fiona Strawbridge (with me chipping in) and Nataša Perović  Profiles page of speakers.

Listen 7m32 s or read below

(Previously posted on the Bloomsbury Learning Exchange blog, 29/3/21) 

The typical child will learn to listen first, then talk, then read, then write. In life, most of us tend to use these abilities proportionately in roughly the same order: listen most, speak next most, read next most frequently and write the least. Yet in educational assessment and feedback, and especially in higher education (HE), we value writing above all else. After writing comes reading, then speaking and the least assessed is listening. In other words, we value most what we use least. I realise this is a huge generalisation and that there are nuances and arguments to be had around this, but it is the broad principle and tendencies here that I am interested in. Given the ways in which technology makes such things as recording and sharing audio and video much easier than even a few years ago (i.e. tools that provide opportunity to favour speaking and listening), it is perhaps surprising how conservative we are in HE when it comes to changing assessment and feedback practices. We are, though, at the threshold of an opportunity whereby our increased dependency on technology, the necessarily changing relationships we are all experiencing due to the ongoing implications of Covid-19 and the inclusive, access and pedagogic affordances of the digital mean we may finally be at a stage where change is inevitable and inexorable.

In 2009 while working in Bradford, I did some research on using audio and video feedback on a postgraduate teaching programme. I was amazed at the impact, the increased depth of understanding of the content of the feedback and the positivity with which it was received. I coupled it with delayed grade release too. The process was: Listen to (or watch) the feedback, e-mail me with the grade band the feedback suggested and then I would return the actual grade and use the similarity or difference (usually, in fact, there was pretty close alignment) to prompt discussion about the work and what could be fed forward. A few really did not like the process but this was more to do with not liking the additional process involved in finding out the grades they had been given rather than the feedback medium itself. Only one student (out of 39) preferred written feedback as a default and this included three deaf students (I arranged for them to receive BSL signed feedback recorded synchronously with an interpreter while I spoke the words).  Most of the students not only favoured it, they actively sought it. While most colleagues were happy to experiment or at least consider the pros, cons and effort needed, at least one senior colleague was a little frosty, hinting that I was making their life more difficult. On balance, I found that once I had worked through the mechanics of the process and established a pattern, I was actually saving myself perhaps 50% of marking time per script though there certainly was some front-loading of effort necessary for the first time.  I concluded that video feedback was powerful but, at that time, too labour- and resource-intensive and stuck with audio feedback for most of the students unless video was requested or needed. I continued to use it in varying ways in my teaching, supporting others in their experimentation and, above all, persuading the ‘powers that be’ that it was not only legitimate but that it was powerful and, for many, preferable. I also began encouraging students to consider audio or video alternatives to reflective pieces as I worked up a digital alternative to the scale-tipping professional portfolios that were the usual end of year marking delight.

Two years later I found myself in a new job back in London and confronted with a very resistant culture. As is not uncommon, it is an embedded faith and dependency on the written word that determines policy and practice rather than research and pedagogy. In performative cultures, written ‘evidence’ carries so much more weight and trust, apparently irrespective of impact. Research (much better and more credible than my own) has continued to show similar outcomes and benefits (see summary in Winstone and Carless, 2019) but the overwhelming majority of feedback is still of the written/ typed variety. Given the wealth of tools available and the voluminous advocacy generated through the scholarship of teaching and learning and potential of technology in particular (see Newman and Beetham, 2018, for example), it is often frustrating for me that assessment and feedback practices that embrace the opportunities afforded by digital media seemed few and far between.  So, will there ever be a genuine shift towards employing digital tools for assessment design and feedback? As technology makes these approaches easier and easier, what is preventing it?  In many ways the Covid-19 crisis, the immediate ‘emergency response’ of remote teaching and assessing and the way things are shaping up for the future have given a real impetus to notions of innovative assessment. We have seen how many of us were forced to confront our practice in terms of timed examinations and, amid inevitable discussions around the proctoring possibilities technology offered (to be clear: I am not a fan!), we saw discussions about effective assessment and feedback processes occurring and a re-invigorated interest in how we might do things differently.  I am hoping we might continue those discussions to include all aspects of assessment from the informal, in-session formative activities we do through to the ’big’, high-stakes summatives.

Change will not happen easily or rapidly, however. Hargreaves (2010) argues that a principal enemy of education change is social and political conservatism and I would add to that a form of departmental, faculty or institutional conservatism that errs on the side of caution lest evaluation outcomes are negatively impacted.  Covid-19 has disrupted everything and whilst tensions remain between the conservative (very much of the small ‘c’ variety in this context) and change-oriented voices, it is clear that recognition is growing of a need to modify (rather than transpose) pedagogic practices in new environments and this applies equally to assessment and feedback. In the minds of many lecturers, the technology that is focal to approaches to technology enhanced learning is often ill-defined or uninspiring (Bayne, 2015) and the frequent de-coupling of tech investment from pedagogically informed continuing professional development (CPD) opportunities (Compton and Almpanis, 2018) has often reinforced these tendencies towards pedagogic conservatism. Pragmatism, insight, digital preparedness, skills development, and new ways of working through necessity are combining to reveal a need for and willingness to embrace significant change in assessment practices.

As former programme leader of an online PGCertHE (a lecturer training programme) I was always in the very fortunate position to collect and share theories, principles and practices with colleagues, many of whom were novices in teaching. Though of course they had experienced HE as students they were less likely to have had a more fossilised sense of what assessments and feedback should or could look like. I also have the professional and experiential agency to draw on research-informed practices not only by talking about them but through exemplification and modelling (Compton and Almpanis, 2019). By showing that unconventional assessment (and feedback) are allowed and can be very rewarding we are able to sow seeds of enthusiasm that lead to a bottom-up (if still slow!) shift away from conservative assessment practices. Seeing some colleagues embrace these strategies is rewarding but I would love to see more.

References 

Bayne, S. (2015). ‘What’s the matter with ‘technology-enhanced learning?’ Learning, Media and Technology, 9 (1), 251-257.

Bryan, C., & Clegg, K. (Eds.). (2019). Innovative assessment in higher education: A handbook for academic practitioners. Routledge.

Compton, M. & Almpanis, T. (2019) Transforming lecturer practice and mindset: Re-engineered CPD and modelled use of cloud tools and social media by academic developers. Chapter in Rowell, C (ed.) Social Media and Higher Education: Case studies, Reflections and Analysis. Open Book Publishers.

Compton, M., & Almpanis, T. (2018). One size doesn’t fit all: rethinking approaches to continuing professional development in technology enhanced learning. Compass: Journal of Learning and Teaching,11(1).

Hargreaves, A. (2010). ‘Presentism, individualism, and conservatism: The legacy of Dan Lortie’s Schoolteacher: A sociological study’. Curriculum Inquiry, 40(1), 143-154.

Newman, T. and Beetham, H. (2018) Student Digital Experience Tracker 2018: The voices of 22,000 UK learners. Bristol: Jisc.

Winstone, N., & Carless, D. (2019). Designing effective feedback processes in higher education: A learning-focused approach. Routledge.

Dr Eva Mol; Dr Martin Compton- summary of paper presented at UCL Education conference 6th April 2022

‘Ungrading’ is a broad term for approaches that seek to minimise the centrality of grades in feedback and assessment. The goal is to enable students to focus on feedback purely as a developmental tool and to subvert the hegemony and potentially destructive power of grades. Fundamentally, ungrading is, at one end of the scale, completely stopping the process of adding grades to student work. A less radical change might be to shift from graded systems to far fewer gradations such as pass/ not yet passed (so called ‘minimal grading’). You don’t fatten the pig by weighing it

In addition to the summary offered in this post we began with the definition above and encouraged colleagues to consider critiques of the existing grading-dominated zeitgeist in terms of reliability, validity and  fairness. Grades become a proxy for learning in the minds of both students and lecturers and huge distractions away from the potentials of feedback and genuine dialogue about the work rather than the percentage or grade letter appended to it.

Grades can dampen existing intrinsic motivation… enhance fear of failure, reduce interest, decrease enjoyment in class work, increase anxiety, hamper performance on follow-up tasks, stimulate avoidance of challenging and heighten competitiveness (Schinske & Tanner, 2014)

We summarised the range of possibilities for those interested from simply talking about threats and potential detrimental effects of grades (as well as perceived benefits) through to wholesale, systemic change.

• Scepticism/ discussion/ dialogue
• Piloting no grades on small or low stakes work
• ‘Conceal’ grades in feedback
• Discussed (even negotiated) grades after engagement with feedback
• Designing out grading
• Students collaborate on criteria
• Grade only for final summatives
• Minimal grading (e.g. Pass/ fail)
• Remove grades for early modules or years
• Students self-grade
• All students graded ‘A’
• Institutional level – no grade policies

One of my ungrading experiences (Eva Mol)

These are based on teaching I did at Brown University (Providence Rhode Island), with a classroom of graduate and undergraduate students from archaeology and philosophy. I decided to give them all an A (highest mark possible) before the class started.

What did I learn about students?

• Initially it was a shock to get students out of the system of marks! For most it is really their only mode of thinking about progress and learning, they wondered why take a course if there was no mark (which I think is very disconcerting).
• However, this shifted quickly from shock to viewing the class as a few hours of relief from the system, followed by less anxiety, more experimentation, and students thinking freely and critically both about the system, as well as what they wanted to achieve in a course.
• Much more engagement with the content of the course material and weekly readings
• Discussions were more lively as there was less performance anxiety, students were more personal as well.
• They set their own personal goals for the class, and I as instructor helped them achieve it. These were a variety of things: speaking at a conference, writing a blog, writing an article. At the end, they realised they got much more out of a course than they ever could with just a mark.

What did I learn as a teacher?

• It is not less work! I still had to read what my students wrote, correspond to emails, give feedback. But it is really different and much more enjoyable work: when not reading in the context of how writing scores against a grading scale, you can allow yourself to appreciate what students accomplished in their writing.
• Comments on feedback were much more rewarding because it was not to justify the mark for the administration, but how you can help students improve, and because there is no mark involved, students read feedback.
• It made me a more engaged instructor, more flexible, creative, and more relaxed.
• Because I could be flexible, I was much better equipped to deal with building in equity and inclusion.
• It also forced me to critically reflect on the relationship between grading and teaching, contextualize how we have normalized the artificial frame of numerical feedback, and look for alternatives aimed at my personal pedagogy.

I felt empowered to question all aspects of the folklore. Why am I assigning a research paper even though it’s always a disappointment? Why do I care whether students use MLA formatting correctly down to the last parenthesis and comma? (I don’t.) Why should I worry about first-year writing as a course meant to prepare students for the rest of college? Why can’t I have autonomy over what I think students should experience? (Warner 2020, 215).

Now is the time

The pandemic showed that we can change, if necessary, perhaps now is the right time to reflect on the system. We have an opportunity to shift the way students feel about their own learning and move away from more traditional words associated with grading.

The classroom remains the most radical space of possibility in the academy. (bell hooks 1994)

—————————————————

References and more about ungrading

bell hooks (1994). Teaching to Transgress: Education as the Practice of Freedom, New York: Routledge.

Blum, S. and A. Kohn (eds.), (2020). Ungrading: Why rating students undermines learning (and what to do instead). West Virginia University Press.

Blum, S. (2019). Why Don’t Anthropologists Care about Learning (or Education or School)? An Immodest Proposal for an Integrative Anthropology of Learning Whose Time Has Finally Come. American Anthropologist 121(3): 641–54.

Eyler, J. R. (2018). How Humans Learn: The Science and Stories behind Effective College Teaching. Morgantown: West Virginia University Press.

Inoue, A.B. (2019). Labor-Based Grading Contracts: Building Equity and Inclusion in the Compassionate Writing Classroom. Fort Collins, CO: WAC Clearinghouse and University Press of Colorado. https://wac.colostate.edu/books/perspectives/labor/.

Rust, C. (2007), Towards a scholarship of assessment, Assessment & Evaluation in Higher Education 32:2, 229-237

Sackstein, S. (2015). Hacking Assessment: 10 Ways to Go Gradeless in a Traditional Grades School. Cleveland, OH: Times 10 Publications

Schinske, J., and K. Tanner (2014). Teaching more by grading less (or differently). CBE – LIfe Sciences Education 13, (2), 159-166

Stommel J., (2017), Why I don’t Grade. JesseStommel.com https://www.jessestommel.com/why-i-dont-grade/

Warner, J., (2020). Wile E. Coyote, the Hero of Ungrading, in S. Blum, Ungrading: Why rating students undermines learning (and what to do instead). West Virginia University Press, 204-218

Wormeli, R. (2018). Fair Isn’t Always Equal: Assessment and Grading in the Differentiated Classroom. 2nd ed. Portland, ME: Stenhouse.

I have been forced to confront a prejudice this week and I’m very glad I have because I have significantly changed my perspective on Team Based Learning (TBL) as a result. When I cook I rarely use a recipe: rough amounts and a ‘bit of this; bit of that’ get me results that wouldn’t win Bake Off but they do the job.  I’m a bit anti-authority I suppose and I might, on occasion, be seen as contrary given a tendency to take devil’s advocate positions.  As a teacher educator, and unlike many of my colleagues over the years, I tend to advocate a more flexible approach to planning, am most certainly not a stickler for detailed lesson plans and maintain a sceptisicm (that I think is healthy) about the affordances of learning outcomes and predictably aligned teaching. I think this is why I was put off TBL when I first read about it. Call something TBL and most people would imagine something loose, active, collaborative and dialogic. But TBL purists (and maybe this was another reason I was resistant) would holler: ‘Hang on! TBL is a clearly delineated thing! It has a clear structure and process and language of its own.’ However, after attending a very meta-level session run by my colleague, Dr Pete Fitch, this week I was embarrassed to realise how thoroughly I’d misunderstood its potential flexibility and adaptability as well as the potentials of different aspects I might be sceptical of in other contexts.

Established as a pedagogic approach in medical education in the US in the 1970s, it is now used widely across medical education globally as well as in many other disciplinary areas. In essence, it provides a seemingly rigid structure to a flipped approach that typically looks like this:

• Individual pre-work – reading, videos etc.
• Individual readiness assurance test (IRAT) – in class multi-choice text
• Team readiness assurance teast (TRAT) – same questions, discussed and agreed- points awarded according to how few errors are made getting to correct response
• Discussion and clarification (and challenge)- opportunities to argue, contest, seek clarification from tutor
• Application- opportunity to take core knowledge and apply it
• Peer evaluation

This video offers a really clear summary of the stages:

Aside from the rigid structure, my original resistance was rooted in the knowledge-focussed tests and how this would mean sessions started with silent, individual work. However, having been through the process myself (always a good idea before mud slinging!), I realised that this stage could achieve a number of goals as well as the ostensible self-check on understanding. It provides a framing point for students to measure understanding of materials read; it offers-completely anonymously- even to the tutor, an opportunity to guage understanding within a group; it provides an ipsative opportunity to measure progress week by week and acts additionally as a motivator to actually engage with the pre-session work (increasingly so as the learning culture is established). It turns a typically high stakes, high anxiety activity (individual test) into a much lower stakes one and provides a platform from which intial arguments can start at the TRAT stage. A further advantage therefore could be that it helps students formatively with their understanding of and approaches to multi-choice examinations in those programmes that utilise this summative assessment methodology.  In this session I changed my mind on three questions during the TRAT, two of which I was quietly (perhaps even smugly) confident I’d got right. A key part of the process is the ‘scratch to reveal if correct’ cards which Pete had re-imagined with some clever manipulation of Moodle questions. We discussed the importance of the visceral ‘scratching’ commitment in comparsion to a digital alternative and I do wonder if this is one of those things that will always work better analogue!

The cards are somewhat like those shown in this short video:

To move beyond knowledge development, it is clear the application stage is fundamental. Across all stages it was evident how much effort is needed in the design stage. Writing meaningful, level appropriate multi-choice questions is hard. Level-appropriate, authentic application activities are similarly challenging to design. But the payoffs can be great and, as Pete said in session, the design lasts more than a single iteration. I can see why TBL lends itself so well to medical education but this session did make me wish I was still running my own programme so I could test this formula in a higher ed or digital education context.

An example of how it works in the School of Medicine in Nanyang Technological University can be seen here:

The final (should have been obvious) thing spelt out was that the structure and approach can be manipulated. Despite appearances, TBL does enable a flexible approach. I imagine one-off and routine adaptations according to contextual need are commonplace.  I think if I were to design a TBL curriculum, I’d certainly want to collaborate on its design. This would in itself be a departure for me but preparing quality pre-session materials, writing good questions and working up appropriate application activites are all essential and all benefit from collaboration or, at least, a willing ‘sounding board’ colleague.  I hope to work with Pete on modelling TBL across some of the sessions we offer in Arena and I really need to get my hands on some of those scratch cards!

[listen -10 mins -or read below]

Questions you ask students are at the heart of teaching and assessment but where and how you ask them, the types of questions you ask and the ways you ask them can sometimes be neglected. This is especially true of the informal (often unplanned) questions you might ask in a live session (whether in-person or online) where a little additional forethought into your rationale, approach and the actual questions themselves could be very rewarding. I was prompted to update this post when reviewing some ‘hot questions’ from new colleagues about to embark on lecturing roles for the first time.  They expressed the very common fears over ‘tumbleweed’ moments when asking a question, concerns over nerves showing generally and  worries about a sea of blank screens in online contexts and ways to check students are understanding, especially when teaching online.  What I offer below is written with these colleagues in mind and is designed to be practically-oriented:

What is your purpose? It sounds obvious but knowing why you are asking a question and considering some of the possible reasons can be one way to overcome some of the anxieties that many of us have when thinking about teaching. Thinking about why you are asking questions and what happens when you do can also be a useful self-analysis tool.  Questions aren’t always about working out what students know already or have learned in-session. They can be a way of teaching (see Socratic method overview here and this article has some useful and interesting comments on handling responses to questions), a way of provoking, a way of changing the dynamic and even managing behaviour.  In terms of student understanding:  Are you diagnosing (i.e. seeing what they know already), encouraging speculation, seeking exemplification or checking comprehension?  Very often what we are teaching- the pure concepts – are the things that are neglected in questioning. How do we know students are understanding?  For a nice worked example see this example of concept checking.

The worst but most common question (in my view). Before I go any further, I’d like to suggest that there is one question (or question type) that should, for the most part, be avoided.  What do you think that question might be? It is a question that will almost always lead to a room full of people nodding away or replying in other positive ways. It makes us feel good about ourselves because of the positive response we usually get but actually can be harmful. The reason for it is that when we ask it there are all sorts of reasons why any given student might not actually give a genuine response. Instead of replying honestly they see others nodding and do not want to lose face, appear daft, go against the flow. They see everyone else nodding and join in. But how many of those students are doing the same? How does it feel when everyone else appears to understand something and you don’t? Do you know what the question is yet? See foot of this post to check** (Then argue with me in comments if you disagree).

Start with low stakes questions. Ask questions that ask for an opinion or perspective or to make a choice or even something not related to the topic. Get students to respond in different ways (a quick show of hands, an emoji in chat if teaching online, a thumbs up/ thumbs down to an e-poll or a controversial quotation – Mentimeter does this well). All these interactions build confidence and ease students into ‘ways of being’ in any live taught session. Anything that challenges any assumptions they may have about how teaching ‘should’ be uni-directional and help avoid disengagement are likely to help foster a safe environment in which exchange, dialogue, discussion and the questions that are at the heart of those things are comfortably accepted. Caveat: it is worth noting here that what we might assume if a student is at the back and not contributing will almost certainly have reasons behind it that are NOT to do with indolence or distraction. A student looking at their phone may be anxious about their comprehension and be using a translator, for example  They are there! This is key. Be compassionate and don’t force it. Build slowly.

Plan your questions. Another obvious thing but actually wording questions in advance of a session makes a huge difference. You can plan for questions beyond opinion and fact checking types (the easiest to come up with on the fly). Perhaps use something  like The Conversational Framework or Bloom’s Taxonomy to write questions for different purposes or of different types.  Think about the verbal questions you asked in your last teaching session. How many presented a real challenge? How many required analysis, synthesis, evaluation? Contrast to the number that required (or could only have) a single correct response. The latter are much easier to come up with so, naturally, we ask more of them. If framing the higher order questions is tough on the spot, maybe jot a few ahead of the lecture or seminar. If you use a tool like Mentimeter to design and structure slide content it has many built in tools to encourage you to think about questions that enable anonymous contributions from students.

The big question. A session or even a topic could be driven by a single question. Notions of Enquiry and Problem-Based Learning (EBL/ PBL) exploit well designed problems or questions that require students to resolve. These cannot of course be ‘Google-able’ set response type questions but require research, evidence gathering, rationalisation and so on. This reflects core components of constructivist learning theory.

The question is your answer. Challenging students to come up with questions based on current areas of study can be a very effective way of gauging the depth to which they have engaged with the topic. What they select and what they avoid is often a way of getting insights into where they are most and least comfortable.

Wait time. Did you know that the average time lapse between a question being asked and a student response is typically one second? In effect, the sharpest students (the ‘usual suspects’ you might see them as) get in quick. The lack of even momentary additional processing time means that a significant proportion (perhaps the majority) have not had time to mentally articulate a response. Mental articulation goes some way to challenging cognitive overload so, even where people don’t get a chance to respond the thinking time still helps (formatively).  There are other benefits to building in wait time too. This finding by Rowe (1974)* is long ago enough for us to have done something about it. It’s easy to see why we may not have done though…I ask a question; I get a satisfyingly quick and correct response…I can move on. But instilling a culture of ‘wait time’ can have a profound effect on the progress of the whole group. Such a strategy will often need to be accompanied by….

Targeting. One of the things we often notice when observing colleagues ‘in action’ is that questions are very often thrown out to a whole group. The result is either a response from the lightning usual suspect or, with easier questions, a sort of choral chant. These sorts of questions have their place. They signify the important. They can demarcate one section from another. But are they a genuine measurement of comprehension? And what are the consequences of allowing some (or many) never to have to answer if they don’t want to? Many lecturers will baulk at the thought of targeting individuals by name and this is something that I’d counsel against until you have a good working relationship with a group of students but why not by section? by row? by table? “someone from the back row tell me….”. By doing this you can move away from ‘the usual suspects’ and change your focus- one thing we can inadvertently do is to focus eye contact, attention and pace on students who are willing and eager to respond thereby further disconnecting those who are less confident or comfortable or inclined to ‘be’ the same.

Tumbleweed.  The worry of asking a question and getting nothing in response can be one of those things that leads to uni-directional teaching. A bad experience early on can dissuade us from asking further questions and then the whole thing develops its own momentum and only gets worse. The low stakes questions, embedding wait time and building a community comfortable with (at least minimal) targetting are ways to pre-empt this. My own advice is that numbers are with you if you can hold your nerve and relaxed smile. Ask a question and look at the students and wait. 30 seconds is nothing but feels like an eternity in such a situation. However, there are many more of them than you and one of them will break eventually! Resist re-framing the question or repeating it too soon but be prepared to ask a lower stakes version and building from there. More advice is available in this easy access article.

Technology as a question not the answer. Though they may seem gimmicky (and you have to be careful that you don’t subvert your pedagogy for colour and excitement) there are a number of in- or pre-session tools that can be used.  Tools like Mentimeter, Polleverywhere, Socrative, Slido, Kahoot all enable different sorts of questions to be answered as does the ‘Hot Questions’ function in Moodle that prompted me to re-post this.

Putting thought into questions, the reason you are asking them and how you will manage contributions (or lack thereof) is something we might all do a little  more of, especially when tasked with teaching new topics or to new groups or in new modalities.

*Rowe, M. B. (1974). Wait‐time and rewards as instructional variables, their influence on language, logic, and fate control: Part one‐wait‐time. Journal of research in science teaching, 11(2), 81-94. (though this original study was on elementary teaching situations the principles are applicable to HE settings)

**Worst question? ‘Does everyone understand?’ or some such variant such as nodding and smiling at your students whilst asking ‘All ok? or ‘Got that?’. Instead ask a question that is focussed on a specific point. Additionally, you might want to routinely invite students to jot their most troubling point on a post it or have an open forum in Moodle (or equivalent space) for areas that need clarifying.

[This is an update- actually more of a significant re-working- of my own post,  previously shared here: https://blogs.gre.ac.uk/learning-teaching/2016/11/07/thinking-about-questions/]

Some colleagues and I were tasked with producing a ‘toolkit’ for other colleagues looking to improve/ optimise engagement. The toolkit can be seen in this online version of the ‘engagement’ toolkit or dowloaded from here in Word format. I hope colleagues find it useful.

What struck me most when talking about and reading around this topic is how problematic it is as a concept and how little time is actually given to deconstructing meaning and principles. We throw words like ‘engagement’, ’employability’ and ‘wellbeing’ (should it be hyphenated?) around, without even checking that we have a shared understanding of what we mean. The same could be said for ‘learning’ and ‘teaching’ too I suppose. For a start, engagement in activity is not a proxy for learning but is easily confused and conflated as such. Back when I was a sessional lecturer in several further education colleges many of the quality assurance processes in every place were informed by the lengthy shadow cast by Ofsted. As I recall, along with ‘differentiation strategies’ and ‘negotiated, individualised outcomes’ (I kid you not) we needed to show that we were on top of student engagement. So, lesson plans were expected to specify student engagement activities and observation forms sought ratings in terms of ‘successful’ engagement. I can imagine people asking ‘what’s wrong with that?’ and I suspect I didn’t question it then to be honest. I was aware that I was constructing an artifice in those plans and observed sessions though. What it tended to do (definitely in my case and certainly later in the case of many of the teachers I observed) was to encourage a cynical acknowledgement of this demand. You end up shoe-horning in activities where engagement (read: students being busy) is visible because the big problem was that only engagement that was in-your-face obvious was likely to count.

This is why I very much like the engagement framework suggested by Redmond et al. (2018) and is represented below. First, it encourages us to conceptualise types of engagement and secondly, and crucially in my view, implores us NOT to find ways to measure engagement but to eschew measurement and focus on developing an environment where different types of engagement are valued and fostered.

Redmond, P., Abawi, L. A., Brown, A., Henderson, R., & Heffernan, A. (2018). An online engagement framework for higher education. Online learning, 22(1), 183-204.

[if you have never seen or used Mentimeter then a quick look here may help]

[Listen -11 mins or read below]

When it comes to tools that will do a teaching and learning job there is a world of dedicated educational technology and ‘productivity’ tools to choose from. I’m very much an experimenter and a fiddler. If I see someone using or referring to a website or tool that looks interesting in a meeting or at a conference, I am there in seconds signing up and playing around and making judgements that have become a staple of my needs and preference-focussed filtration system. In broad terms I like to be able to try things for free, for it to be relatively intuitive and straightforward and (most important) fit for either pre-defined or imagined teaching or assessment purposes. I have written more about the how and why of this with my former colleague Dr Timos Almpanis here and in chapter 4 of this collection. I try not to evangelise and I am very much of the school that would argue that purpose rather than any given tool should be a starting point for discussions about integrating digital approaches but there’s something about what Mentimeter can do and how it does it that means I do sometimes slip into ultra-enthusiast mode.  Unlike a lot of tech approaches and tools that pass the initial ‘free, easy, fit for purpose’ test there’s something about the breadth of purpose that Mentimeter is fit for and its intuitiveness that, for me, make it a class above other tools. Also see here for why Chris Little at Keele made this point a while back and here for an evaluation of a number student response tools including Mentimeter from when I and a colleague at Greenwich were tasked with identifying  what the best institution-wide student response system would be.

1. It’s not hardware dependent

Like a lot of people similarly enthusiastic about opportunities for enhancing student interaction and engagement with digital technologies, I spent a lot of time (much of which was ultimately wasted) focussed on hardware. From interactive whiteboards to in-class ipad sets to PDAs and ‘flipcams’ the issues that directly impeded scaling of use as well as my own enthusiasm  were related to one or more of the following:

• Amount of training needed
• Device security and storage
• ‘Just in time’ access limits
• Responsibility for maintenance
• Rapidity of obsolescence of kit

In my view, all these were factors that afflicted ‘clickers’ (voting pods that were handed round in face to face sessions) – as revolutionary as they promised to be -they were only ever used by the few, despite the gleaming aluminium cases and the sumptuous foam inserts that the clicker devices sat in. The BYOD dependence on user devices when it comes to cloud-based software alternatives like Mentimeter means that:

• People usually know how to use their own devices or at least access the internet
• Device security, maintenance, updating is not an issue
• They are, by definition, available; turning an oft-cited teacher frustration of mobile device distraction into a potential virtue

‘What if students don’t have a device?’ is a common question but, like many things in this domain, it’s largely about framing. I will always make participation optional and make it clear ‘if you have a device on you’ if in a face to face setting or ‘if you have a big enough screen or a separate device nearby’ if online and frequently subvert the assumption that responses need to be individual and precede voting with group-based discussion with one person per group responding.

I have moved between institutions in the last year and both have invested in a site licence and access to the full suite of tools and functionality Mentimeter offers. This privilege is something that must be acknowledged so it’s certainly not ‘free’ any more (though I  personally pay nothing of course!) the free version is still relatively generous. In my view it’s an exemplary freemium set up. Just on the right side of frustrating in amongst the persuasive.

2. It’s a slide/ presentation tool that has many merits in its own right

One thing that is often missed because the ‘point’ of Mentimeter is interaction is how well it works as an alternative to PowerPoint as a presentation tool.  Even though PowerPoint remains the default across higher education for slide production (even during that weird period when everyone was doing Prezis!), for the most part colleagues seem to struggle to break from the desktop app habit. As a consequence, sharing of slides becomes an upload/ download faff or, even if sharing is managed via MS Office cloud storage there are often restrictions on who can view. Mentimeter generates a link so the first benefit is slides can be shared as easily as any website link. Secondly, the participation link enables the students to see the slides (including detail of pictures) on their own devices in real time (as well as or possibly INSTEAD OF the main screen). Thirdly the author interface is simple, there is a variety of slide types and styles, the copyright free image gallery is easy to use as is the ALT text prompt. Fourthly, the ability to add simple interactions (eg thumbs up or down) mean that students can be invited to contribute even to content delivery type slides by, for example, agreeing or disagreeing with a controversial idea or quotation.  The slides have more limited space for text and this (to some a limitation) is an excellent discipline when preparing slides to minimise the text and challenge the tendency many of us have to use too much of it.

3. The participation and interaction options are substantial and adaptable

In a previous post there were a few occasions where students chose notoriety over maturity and tried to undermine sessions by being abusive in open text questions. This led to something of a knee-jerk response by some colleagues who questioned whether the tool should be supported or used at all. Much like (way way back) access to YouTube was banned for all students AND teaching staff in a college I worked in because ONE student accessed a (seriously) inappropriate video. The sledgehammer / nut response was not the way to address things, not least because Mentimeter’s existing tools and functionality enable users to avoid and tackle such behaviours. So, if open text questions are used there are ways of monitoring and filtering content (including a profanity filter) and of the ten interaction/ question types only three are open text.  To grasp this, however, does often necessitate more than superficial exploration and experimentation (or coming to one of my hour-long workshops!) One thing I commonly do is encourage colleagues to consider how they might eschew the favoured word cloud and open text formats and find ways of fully exploiting the lesser used types.  In addition, it’s important to think about how the interactions are presented and managed. A well-designed question can be an excellent vehicle for prompting discussion prior to ‘voting’ or as a prompt for analysing/ rationalising responses that have already been offered.

4. Frequent updates and improvements

There’s no resting on laurels with Mentimeter and there does seem to be acknowledgement of user requests. For example, the ability to embed video in slides from YouTube is a real blessing and, if using Mentimeter as a slide tool as well as for interaction, further minimises shifting between tabs or different software.  The recently introduced collaborative authoring of presentations was much requested at UCL and enables more efficient working in addition to the collaborative potential. A very recent and welcome improvement is the ability to have active hyperlinks (in both participation and presentation modes). The ‘Mentimote’ tool that allows you to use your smartphone as a slide clicker, moderation tool and presentation embellisher has also recently switched from beta to ‘fully fledged’ mode and works very well, especially for live in-person events.

5. When Covid came, Mentimeter was equipped to adapt.

The default pace setting in Mentimeter is ‘presenter paced’. That is, the presenter advances slides and only then can participants see them. This is very much in keeping with the how Mentimeter (presumably) was conceived and how many people who are users regard it. However, the non default option (audience paced) allows slide collections with interactions to be accessed at audience pace. When lessons switched online almost across the board it was common for academic colleagues to take the intuitive approach and try to replicate face to face teaching in online environments via Zoom, Teams or Collaborate. They often tried to incorporate Mentimeter slides too. Whilst this is do-able and it is something I routinely use myself, the complexity and both mental and actual bandwidth this layer added to already struggling staff and students (with kit, with space, with implications of Covid) meant that it often felt unsatisfactory. Alongside my and colleagues’ recommendations to rethink how online time could be exploited and optimised I encouraged colleagues to think about the possibilities of using Mentimeter asynchronously. By encouraging participation ahead of a session then presenting results in a session much faffing, device and screen changing is removed but still students have a buy-in to the content. When I came to my current post it was fascinating to see how colleagues in similar positions to my own such as Dr Silvia Colaiacomo were saying the same thing here.

If you want to read more on my thoughts about Mentimeter see this post and also this collaboration with two former colleagues (Dr Gerhard Kristandl from Greenwich and Paramedic extraordinaire Richard Ward who is at Cumbria).

Here, too, is a video case study I made with a colleague and student from the Division of Psychiatry on academic and student use of Mentimeter.

Colleagues at UCL interested in using Mentimeter start here: https://blogs.ucl.ac.uk/digital-education/2020/07/09/mentimeter-at-ucl/

The recently published special edition compendium from JLDHE of reflections on the impact of Covid19 on higher education teaching, learning and assessment is an excellent, accessible and diverse resource. The range and quality of articles makes me feel quite proud to be a part of it! The contents page can be accessed here. It is an open access journal and each article uses a common format under strict wordage guidelines so it really is possible to dip in and out.

My article is here and I offer an audio version of it below for those that prefer to listen or who may gain something from my efforts at being expressive.

In that role I was working closely with Dr Alison Gilmour (now at UWS) and I would also recommend her piece on ‘Adopting a pedagogy of kindness’.

[listen 8m 4s or read below]

Warning: If you dislike mangled, mixed and over-extended metaphors, please place this safety notice back in the pocket in front of you, remove shoes and exit swiftly (or parachute to possible co-pilot roles and responsibilities below).

Once you have navigated through the sea of ‘keep out’ and other similar warning signs and committed to dipping your toes into the shark-infested hybrid waters (Whatever you call it, we’re talking simultaneous live face to face and online teaching- Zoom or Teams mediated), you will know that number 1 in the advice chart is: ‘Get yourself a co-pilot’. But what is a co-pilot? What role could or should they play? Should they, like the kind of co-pilot you get on a plane, be able to land the thing if you were to become suddenly incapacitated? Is it a good idea to recruit students to do it? The following suggestions are based on doing the hybrid thing a few times a year over the last 3 or 4 years and a dozen or so times in the last 3 or 4 weeks.

Do you even need a crew? Or will a passenger do?

So, you have your pilot’s licence and you want to get into the cockpit asap. Surely the last thing you need is someone else twiddling knobs, making unauthorised announcements and looking better in their reflector shades? There are all sorts of reasons why flying anything from the new Boeing Jumbo Lecturehall to the Cessna 6-seater Skyhawk Seminar is likely to be something you want or are obliged to do alone. Assuming for a moment that your airline (read: institution, faculty/ department) can fund a co-pilot then there are plenty of reasons why you might benefit from having someone by your side. If the plane is no private jet and there’s not the reserves to dip into you might still want to find a way to get someone in and give them a shiny badge with ‘co-pilot’ written on it, whether or not they could land the thing themselves. Whatever the size of your aircraft this is more than the flying you may (or often not) have been trained to do. Suddenly you’re being asked to carry on flying but also remote control a car (perhaps with all your loved ones in it) on the motorway 30,000 ft below you. How helpful a co-pilot can be will depend on a range of factors:

• Their own professional training and subject expertise
• Whether they are paid
• Whether this is likely to be a one off event or a sustained new way of travelling
• All the variables you can imagine

So in an ideal world an equivalently trained co-pilot is optimal (a close colleague perhaps to get you through the first time), an in-training colleague (such as a PGTA), a student quality reviewer (UCL colleagues look out for an announcement on this soon) or even one or more of the passengers (sorry, students) you are flying that day. Please note: In the latter case, you will need to think through the implications, limit or spread the load and note that co-pilots have reported intense concentration requirements leading to them not processing what the pilot has been saying. Whatever your situation and whoever you are able to work with the following are roles that you might want to consider for the co-pilot. Most important is to agree and delineate roles prior to the session starting.

(possible) Co-pilot roles and responsibilities

Most of the following would not be reasonable to expect from a student volunteer and ALL of them would be unreasonable for any co-pilot, however well they were remunerated! Where students are asked to help maintain connection between online and in-person participants it may be prudent not to think in terms co-piloting given the necessary limits to what they can be expected to do. In those circumstances it may be wise to have a few students with different responsibilities (eg. One simply signals when an online hand is raised, another paraphrases in chat the in-person contributions).

Pre-flight checks

• Helping to check equipment works ahead of a session
• Role playing as a student in physical or online spaces to identify potential issues (from font size on display which may be reduced due to additional elements needed on screen to audio levels of mics and speakers)
• Sending reminders, fielding questions about joining instructions and/ or seeding persistent back-channel (if used)
• (Immediately before take off) Participate in final checks, guide students to seats (for optimal acoustics), assist with registration
• Post re-assuring notes in online space and/ or backchannel- tell remote attendees when to expect pilot’s mic to be unmuted
• Reminding in-person students to mute both mics AND speakers on their devices if they access the remote participation link (note: the more in-person students who join the online space, the harder it is to set up breakout groups)
• Have links to all in-session and post-session resources to hand to paste into chat or backchannel
• Serving drinks is not advised

Take-off

• Welcoming in chat the online participants, by name if possible if group is not too large
• If appropriate, facilitate connections between the two modalities- e.g. Volunteer students in the physical space waving over your shoulder into your webcam and the online particpants wave back.
• Asking in chat if audio can be heard when pilot turns on their mic and be the point of contact if any technical issues are experienced
• Reiterating participation protocols for contributions from online participants in chat (e.g. Raise electronic hand); how invitations to speak will work
• Sharing links to or guidance about other channels of participation (e.g. Mentimeter participation link; PowerPoint Live presentation link and how to start simultaneous translation; guidance on starting subtitles in online platform)

Climb & Cruise

• Flag ‘hands up’ to pilot at appropriate junctures/ pre-agreed times
• Paraphrase in-room contributions in chat, especially where audio issues are known. Identifying the contributor (even initials only) helps online participants keep track of things
• (If agreed) respond to questions arising in chat or at least acknowledge that questions have been received/ seen
• Summarise/ theme remote contributions and questions
• Open/ close and monitor breakout rooms
• Send messages to breakout rooms that mirror in person instructions
• In text-only collaborative spaces, add comments if appropriate and smooth facilitation between remote and in-person students

Descent and landing

• Give opportunity to pose questions and commit to sharing responses if not addressed in session
• Share links to resources, activities, follow up work or tasks to be completed ahead of future sessions
• Seek evaluative comments (on connection/ access/ accessibility etc.)
• Signpost preferred media for communications between live events and ahead of next session and where/ how to access any session recordings

What have I forgotten? What have you tried that smooths hybrid/ hyflex sessions with or without a co-pilot? Let me know!

“The Hitchhiker’s Guide to the Galaxy has this to say on the subject of flying. There is an art, it says, or, rather, a knack to flying. The knack lies in learning how to throw yourself at the ground and miss.” Douglas Adams

[listen 12m 35s or read below)

With apologies to both Bridget Fonda and to the killer on the left/ right etc, this was the title of my keynote at Kingston’s ‘Future of Learning’ conference in June 2021 and revamped for FRA in January 2022 and below is my related tuppenceworth in the form of 10 provocations on what we ought to be doing and avoiding as we prepare for at least one more year of uncertainty and change in higher education. I have been teaching online for 5+ years and integrating all things digital for a lot longer. Another way of saying that is I have learnt a lot of things the hard way by getting them wrong, wasting efforts unnecessarily and realising, too slowly, things I’d probably now consider to be obvious.  Everything I say needs to be set against that background. My position and my perspectives are informed by this. I did (and continue to), however, spend a lot of time supporting colleagues from across the disciplines and professional services and across the digital confidence, capabilty, enthusiasm and kit spectrums during and subsequent to the pivot online.

1. “Forced compliance” of 20-21 shouldn’t define how we understand online/ blended/ hybrid (or whatever the heck it’s called now)

When I think about the last year it’s a real mixed bag. If I can push aside the awful bits and focus on the work, I still find that the reflections are necessarily blurred with family/ personal stuff, not least because my daughter (age 9) was at home for a lot of it. She and I worked at either ends of the kitchen and my day was salt and peppered with notes like these asking me to keep the volume down.  This is emblematic in my mind of the necessarily different ways of working and being to what was ‘normal’. This tiny inconvenience is insignificant compared to some of the impacts felt across the sector by many academic staff and students, often struggling to find adequate space and kit to work with.  The suddenness and the necessity combined to create a situation where the decision to teach in these ways was largely out of our hands. Many have argued that the silver lining of the dark cloud that is still very much part of our lives is the way the pandemic catalysed advances in digital confidence and competence unimaginable in ‘normal’ times. The other side of this of course is  how the experiences of some teachers and students tainted many people’s perceptions of what an online or blended education could be.

2. Be (REALLY) honest with ourselves about what worked, what didn’t, what we developed and what we still need

This (for me at least) is one of those things that sounds easy and obvious but is actually quite challenging. Do we ever give ourselves enough time to properly think? In the context of point 1 above, can we properly think without some form of scaffold  and/ or mediation? Do we have enough information to think usefully (and with opportunities for discursive exploration) to reflect with one eye looking back and another looking forward? This image really struck me during the lockdown. It’s from an academic in Canada (Dr. Verena Roberts: image shared with permission) who shared a picture of her son on his ‘reflection rock’. Time for reflection being built into the lockdown school day was a powerful reminder from a very different context of how learning does and should happen beyond those times we are ‘in control’.  So much was represented in that image: What is possible when we relinquish control; valuing the affective; encouraging liberty, creativity and independence. We need to give our students space and tools to reflect but not forget either that we also need the same opportunities.

3. We need to push against binary thinking and simplistic narratives

Those that are not so good at reflecting or embracing nuance or those with strong vested interests in a swift return to ‘normal’  may be inclined to push, echo or sympathise with simplistic narratives: online = BAD; in person = GOOD. From the very start of the pandemic I heard (and almost certainly said myself) ‘we are not the Open University’ but that shouldn’t stop us from being open to models they have developed and honed over decades of offering remote and online education. Respecting and heeding wisdom from OU colleagues and practise, then or looking ahead, does not mean we want to occupy their ground or to become them. To suggest that as a way of closing down discussion is, to my mind, often a symptom of exclusionary thinking. Widening participation means developing an understanding of some of the positive aspects remote connection enabled for significant numbers of our students. By casting light on possibilities otherwise masked by years of convention we have revealed what is possible and the after-image of that has burned deep however swiftly we try to turn off the light.  The best we can offer is impacted by so many variables but we do know that disciplinary contexts help define what we do; that one size solutions would only ever be a temporary fix and that online and remote options in education exist for a reason. Accepting this in reasoned discussions about what post-pandemic pedagogies look like is essential.

4. Lectures and exams. Always elephants in the room; only a lot bigger now

I expect most have seen before this classic 14th century image (a screen grab from a very early iteration of Lecturecast) of students snoozing or chatting in the lecture and the one being distracted by a smart phone under the bench. They think Henry hasn’t noticed of course but it’s amazing what you can see from the front. The pandemic has certainly given us pause for thought about the lecture as default teaching modality (in many places) and, likewise, the exam as default summative assessment (in many places) has been challenged, not least by emerging evidence of attainment gaps closing due to assessment modifications.

I am NOT advocating (like some) the end of the lecture completely BUT it is hard to argue that content-dense, one size fits all pedagogies AS DEFAULT really help us realise the EDI ideals that institutions present with great fanfare as defining the what University  X or Y is all about.

What I am an advocate of is (cf. point 3 above) a push back against binary thinking on the often polarising debate about lectures and exams, for much more nuanced thinking and openeness to new ways of working. Of course, if we conclude that breaking with convention would be better then we need to work out ways to resource and support that. Failing to change because a. we have always done it that way or b. it’s too expensive are fallacious arguments because they appeal to tradition and cost (according to one of my teachers from secondary school). If it’s worth doing then we need to work out how to do it.

5. Replicating face to face models is not optimising affordances of digital stuff

This image is called ‘correspondence cinema’ and is one of a series of cigar box inserts and post cards created between 1899 and 1910 that imagined the year 2000. Most were way off but a few hit the mark- this one I like because it foreshadows Zoom and Teams but also reminds me that one of the big OBVIOUS conclusions many reached at the start of the pandemic was how we could use these tools to replicate what was being lost in lockdown- the face to face session. For lots of reasons though this is less than optimal and is NOT actually a feature of well- planned, invested in, developed online programmes. What should the features be then? Well, of course, there’s a long answer to that but: Along with more emphasis on asynchronous ‘content’ and optimising live, connection time for interaction, dialogue, debate we need to think carefully about our goals, our contexts and where we can use digital options can offer something different, new, better or can give improved access.

6. Start from a position of compassion, trust and openness

A starting point, now more than ever, needs to be one of compassion and trust. We and our students are dealing with the trauma of pandemic. We (and they) have adapted to new pedagogic approaches and will now be adapting once again. At a local level this means building in time for community building and building on the affordances offered by both analogue and digital ways of connecting, supporting and simply ‘being there’. More broadly I would argue that we need to resist narratives that divide; push back against the us and them framing of lecturers and students; resist too the pressures from enterprises that profit from us surveilling our students. Plagiarism detection wasn’t a thing when I was student- I am sure copying was – but the world somehow muddled through. I think it gives a false sense of rigour and can make us less likely to employ tactics previously used (for example, ad hoc mini vivas).

7. Grading can be degrading: ‘Ungrading’ and/or digital assessment and feedback can have powerful impacts

I talk about this here so will resit the urge to thump this tub again!

8. Shiny things (AI, VR, robotics, IoT) are a thing but not THE thing!

I see part of my role as trying to re-assure those colleagues who sometimes feel like the expectations are way up there in terms of what they should be doing- in some ways ‘shiny things’ can be distractors (most are NOT doing this stuff) and actually act as de-motivators. Innovation should be seen as ipsative; that is, it is relative to your own prior approaches. I talk more about his here.

9. Why so much writing? Value multi-modal, value listening, value silence. 

In this post I ask why it is that the communication skill most of us use least (IRL) is the one that is overwhelmly pre-eminent in our (UK) education system. What will it take to offer more audio-visual options and content? My own videos are often rough round the edges though I tend to favour audio options over video anyway.  Also, as someone who talks a lot and is very opinionated, I know how hard it is to factor in thinking and wait time when you’re on a roll and, for many, that difficulty was only exacerbated in remote, online interactions. This is one area where (assuming we are compelled by the need to value listening and silence more) is something we can blunt force train ourselves to do.

10. Value the student voice but don’t confuse want with need

I hope this last one speaks for itself- sometimes what students (indeed, all humans) WANT is not necessarily what they NEED in a given context. Before you ‘ah ha!’ me by referring me back to point 6 above, I would re-iterate that I am stressing here the value of disciplinary AND pedagogic expertise. I’m definitely not saying we always know best but, in the same way a patient might have some ideas about what is troubling them because they can use Google or know someone with similar symptoms does not mean medical professionals ignore their own expertise. If students are pushing back against well thought through approaches or loud voices are demanding X when you have planned Y it may be because it’s not what they expect or hoped for.  Maybe, therefore, we can do more to rationalise our approaches and share the pedagogic approaches we are taking. Anything that gets us talking with one another and with our students about what we are doing and why is likely to help.