‘Deploying AI agents’ sounds so hi tech and futuristic to (non Comp-Sci) me whilst weirdly also resonating of classic 60s and 70s TV shows I loved as a kid. I have been fiddling for a while on the blurred boundaries between LLMs and Agents, notably with Claude, but what appealed when I first saw Manus was the execution of outputs seemingly beyond what Claude can manage. Funnily enough it looks quite a bit like Claude but it seems it is actually a multi-tool agent. I pretty much concur with the conclusion from the MIT Tech review:
While it occasionally lacks understanding of what it’s being asked to do, makes incorrect assumptions, or cuts corners to expedite tasks, it explains its reasoning clearly, is remarkably adaptable, and can improve substantially when provided with detailed instructions or feedback. Ultimately, it’s promising but not perfect.
Anyway, I finally got in, having been on the Manus waitlist for a while. Developed by Chinese startup Monica, it is an autonomous AI agent capable of executing complex online tasks without ongoing human input and created something of a buzz. TL:DR: This is the initial output from first prompt to web-based execution. The selection and categorisation need honing but this in my view is an impressive output. The second version after addition of a follow up prompt.
Longer version:
I wanted to see what I could get from a single prompt so decided to see if it could build a shareable, searchable web page that curates short how-to videos (under five minutes) by higher education educators demonstrating uses of Generative AI. I began by requesting Manus to collect and cluster videos showing how AI is applied in teaching, assessment, feedback, and research (Natural Language Prompt). Manus responded immediately by creating a structured project directory and initiating web searches to identify relevant video content, starting with collections from institutions like Notre Dame and Harvard (which it didn’t get beyond in the first iteration).
Once videos were sourced, Manus automatically filtered them to ensure they were under five minutes in length (but failed to note that one wasn’t a video and others linked to same page where they were already curated!) and produced by educators in the higher education sector (this it did get right). It then categorised them by thematic area (Teaching, Assessment & Feedback, AI Literacy, and Research but the categrosations were not always spot on) while also tagging institutional affiliations (mostly OK). The tagging and filtering work pretty well but the actual sourcing is very limited in that first iteration. A metadata database was created to support an interactive interface, allowing users to search videos by presenter, title, or description and filter by category or institution. Manus built this interface and deployed the collection to a live web page: https://cjubrvkx.manus.space. It took about 15 minutes. Iteration is by natural language prompting for fine tuning.
The initial prompt from me and the start of the executaiton of the processes
After launch, I expanded the scope by asking Manus to include contributions from additional individuals and institutions- I threw in my old buddy Dr. Gerhard Kristandl (Dr GeeKay) who I know has a YouTube channel, I added me- Martin Compton (without any additional info) and it found relevant content from each of us. The other three potential souces I threw in reaped no reward though. This is the reasoning:
I searched for content from Tadhg Blommerde, University of Sydney, and Monash University as well, but most of their videos exceeded our 5-minute duration criteria. I’ve documented all findings in the data directory for reference.
Manus resumed its asynchronous task execution, searching for and integrating further video content into the existing framework enabling me to publish a second version . So the output works, I would need much more thoughtful prompting to get it how I want it and, tbh, there are better ways of curating YouTube content but the principle is what interested me and the output is remarkable given the stages and processes it went through.
I did a session earlier today for the RAISE special interest group on AI. I thought I’d have a bit of fun with it 1. because I was originally invited by Dr. Tadhg Blommerde (and Dr. Amarpreet Kaur) who likes a heterodox approach (see his YouTube channel here) and 2. Because I was preparing on Friday evening and my daughter was looking over my shoulder and suggesting more and more memes. Anyway, I was just reading the chat back and note my former colleague Steve asked: “Is the rest of the sector really short of memes these days now that Martin has them all?” I felt guilty so decided to share them back.
My point: There’s a danger we assume students will invariably cheat if given the chance. This meme challenges educators to reconsider what they define as cheating and encourages transparent, explicit dialogue around academic integrity. What will we lose if we assume all students are all about pulling a fast one?
My daughter (aged 13) suggested this one. How teachers view ChatGPT output: homogenised, overly polished essays lacking individuality. My daughter used the ‘who will be the next contestant on ‘The Bachelor’ (some reality show I am told) image to illustrate how teachers confidently claim they can spot AI-generated assignments because “they all look the same.” My point: I think this highlights early scepticism about AI-produced writing but that we should as educators consider the extent to which these tools have evolved beyond initial assumptions and remind our students (and ourselves) that imperfections and quirks can define a style. Just ask anyone reading one of my metaphor-stretched, overly complex sentences. Perhaps, for too long we have over-valued grammatical accuracy and formulaic writing?
My point: It’s not just about AI detectors of course. It’s more that this is an arms race we can’t win. If we see big tech as our enemy then fighting back with more of their big tech makes no sense. If we see students as the enemy then we have a much bigger problem. Collective punishment and starting with an assumption of guilt are hugely problematic in schools/ unis much as they are in life and tyrannical societies in general. When it comes to revisiting academic integrity I am keen discuss what it is we are protecting. I am also very much drawn to Ellis and Murdoch’s ‘responsive regulation’ approach. I don’t think I’m quite on the same page regarding automated detection but I do agree regarding the application (and resourcing of) deserved sanction for the ‘criminal’ (willful cheats) along with efforts to widen self-regulation and move as many students as possible from carelessness (or chancer behaviours) to self-regulation is critical.
Pretty obvious I guess but my point is this: We also need to resist assumptions that all students prioritise grades over genuine learning and creativity. Yes, there are those who are wilfully trying to find the easiest path to the piece of paper that confirms a grade or a degree or whatever. Yes, there are those whose heads may be turned by the promise of a corner-cutting opportunity. But there are SO many more who want to learn, who are anxious because they know others who are being accused of using these tech inappropriately (because, for example, they use ‘big’ words… really, this has happened). ALSO, we need to challenge the structural features that define education in terms of employability and value. I know how to use chatgpt but I am writing this. Why am I bothering writing? Because I like it. Because – I hope- my writing, even when convoluted (much like this sentence) is more compelling. Because it’s more gratifying than the thing I’m supposed to be doing. Above all, for me, it’s because it actually helps me articulate my thoughts better. We must continue valuing intrinsic motivation and the joy students derive from learning and creating independently. But more than that: we need to face up to the systemic issues that drive more students towards corner cutting or willful cheating. By the way, I often use generated text in things I write. All the alt text in these images is AI generated (then approved / edited by me) for example.
This leads me to the next one. I mean I do use AI every day for translation, transcription, information management, easing access to information, reformatting, providing alternative media, writing alt text… Many don’t I know. Many refuse; I know this too. But we are way into majority territory here I think. Students are recognising this real (or imagined) hypocrisy. The only really valid response to this I have heard goes something like: ‘I can use it because I am educated to x level. first year undergrads do not have the critical awareness or developed voice to make an informed choice’. I mean, I think that may be the case to an extent or in some cases but it reminds me a bit of the ‘pen licences’ my daughter’s primary school issued: you get one when you prove you can use a pencil first (little Timmy, bless him, is still on crayons). Have you seen the data on student routine use of generative AI? It elevates the tool to some sort of next level implement but is it even? I think I could make a better case for normalisation and acceptance of a future where human / AI hybrid writing is just how it is done (as per Dr Sarah Eaton’s work- note the firve other elements in the tenets.)
My point: The narratives around essential changes we need to implement ‘because of AI’ presents a false dichotomy between reverting to traditional exam halls or relying solely on AI detection tools. Neither option adequately addresses modern academic integrity challenges. Exams can be as problematic and inequitable as AI detection. It is not a binary choice. There are other things that can be done. I’ll leave this one hanging a bit as it overlaps with the next one.
Modifying or changing assessment ‘because of AI’ always feels like it feeds ‘us and them’ narratives of a forthcoming apocalypse (already predicted) and couches the change as necessary only because of this insidious, awful thing that no-one wants except men in leather chairs who stroke white cats.
It is of course MUCH more complex than that and much of the desired change has been promoted by folk with a progressive, reform, equity, inclusion eye who do (or immerse themselves in) scholarship of HE pedagogy and assessment practices.
Anyway, a colleague suggested that we should have a collection of ideas about practical ways assessments could be modified to either make them more AI ‘robust’ or at least ‘AI aware’ or ‘ AI inclusive’ (I’m hesitant to say ‘resitant’ of course). Whilst colleagues across King’s have been sharing and experimenting it is probably true to say that there is not a single point of reference. We are in King’s Academy working on remedying this as part of the wider push to support TASK (transforming assessment for students at King’s) and growing AI literacy but first I wanted to curate a few examples from elsewhere to offer a point of reference for me and to share with colleagues in the very near future. I’ve gone for diversity from things I have previously book marked. Other than that, they are here only to offer points of discussion, inspiration, provocation or comparison!
Before I start I should remind KIng’s colleagues of our own guidance and the assessment principles therein, note that with collleagues at LSE, UCL and Southampton I am working on some guidance on the use of AI to assist with marking (forthcoming and controversial). Some of the College Teaching Fund projects looked at assessment and This AI Assessment Scale from Perkins et al. (2024) has a lot of traction in the sector too and is not so dissimilar from the King’s 4 levels of use approach. It’s amazing how 2023 can feel a bit dated in terms of resources these days but this document form the QAA is still relevant and applicable and sets out broader, sector level approarpriate principles. In summary:
Institutions should review and reimagine assessment strategies, reducing assessment volume to create space for activities like developing AI literacy, a critical future graduate attribute.
Promote authentic and synoptic assessments, enabling students to apply integrated knowledge practically, often in workplace-related settings, potentially incorporating generative AI.
Move away from traditional, handwritten, invigilated exams towards innovative approaches like digital exams, observed discipline-specific assessments or oral examinations
Follow guiding principles ensuring assessments are sustainable, inclusive, aligned to learning outcomes, and effectively demonstrate relevant competencies, including appropriate AI usage.
I’m also increasingly referring to the two lane approach being adopted by Sydney which leans heavily into similar principles. Context is different to UK of course but I have a feeling we will find ourselves moving much closer to the broad approach here. It feels radical but perhaps no more radical than what many, if not most, unis did in Covid.
Evaluation of coursework assessments to determine susceptibility to generative AI and potential integration of AI tools.
Redesign of assessments to explicitly incorporate evaluation of ChatGPT-generated outputs, enhancing critical evaluation skills and understanding of AI limitations.
Integration of generative AI within module curricula and teaching practices, providing formative feedback opportunities.
Collection of student perspectives and experiences through questionnaires and focus groups on AI usage in learning and assessments.
Shift towards rethinking traditional assessment formats (MCQs, SAQs, essays) due to AI’s impact, encouraging ongoing pedagogical innovation discussions.
Gamification: Complex immunology concepts taught through a Star Wars-inspired, game-based approach.
AI-driven game design: ChatGPT 4.0 used to structure game scenarios, resources, and dynamic challenges.
Visual resources with AI: DALLE-3 employed to create engaging imagery for learning materials.
Iterative AI prompting: An innovative method using progressive ChatGPT interactions to refine complex game elements.
Practical, collaborative learning: Students collaboratively trade resources to combat diseases, supported by iterative testing and refinement of the game.
Integration of AI: The original essay task was redesigned to explicitly require students to use an LLM, typically ChatGPT.
The change: Individual component of wider collaborative task. Students submit both the AI-generated output (250 words) and a critical evaluation of that output (250 words) on what is unique about a business proposal.
Critical Engagement Emphasis: The new task explicitly focuses on students’ critical analysis of AI capabilities and limitations concerning their business idea.
Reflective Skill Development: Students prompted to reflect on, critique, and consider improvements or extensions of AI-generated content, enhancing their evaluative and adaptive skills.
“Empirical studies suggest that a majority of students cheat. Longitudinal studies over the past six decades have found that about 65–87% of college students in America have admitted to at least one form of nine types of cheating at some point during their college studies”
(Yu et al., 2018)
Shocking? Yes. But also reassuring in its own way. When you are presented with something like that from 2018 (ie. pre chatgpt) you realise that this is not a newly massive issue; it’s the same issue with a different aspect, lens or vehicle. Cheating in higher education has always existed, but I do acknowledge that generative AI has illuminated it with an intensity that makes me reach for the eclipse goggles. There are those that argue that essay mills and inappropriate third party support were phenomena that we had inadequately addressed as a sector for a long time. LLMs have somehow opened a fissure in the integrity debate so large that suddenly everyone wants to do something about it. it has become so much more complex because of that but also that visibility could be seen positively (I may be reaching but I genuinely think there is mileage in this) not least because:
1. We are actually talking about it seriously.
2. It may give us leverage to effect long needed changes.
The common narratives I hear are ‘where there’s a will, there’s a way’ and chatgpt makes the ‘way’ easier. The problem though, in my view, is that just because the ‘way’ is easier does not mean the ‘will’ will necessarily increase. Assuming all students will cheat does nothing to build bridges, establish trust or provide an environment where the sort of essential mutual respect necessary for transparent and honest working can flourish. You might point to the stat at the top of this page and say we are WAY past the need to keep measuring will! Exams, as I’ve argued before, are no panacea, given the long-standing issues of authenticity and inclusivity they bring (as well as being the place where students have shown themselves to be most creative in their subversion techniques!).
In contrast to this, study after study is finding that students are increasingly anxious about being accused of cheating when that was never their intention. They report unclear and sometimes contradictory guidance, leaving them uncertain about what is and isn’t acceptable. A compounding issue is the lack of consistency in how cheating is defined. it varies significantly between institutions, disciplines and even individual lecturers. I often ask colleagues whether certain scenarios constitute cheating, deliberately using examples involving marginalised students to highlight the inconsistencies. Is it ok to get structural, content or proof reading suggestions from your family? How does your access to human support differ if you are a first generation, neurodivergent student studying in a new language and country? Policies usually say “no” but to fool ourselves that this sort of ‘cheating’ is not routine would be hard to achieve and even harder to evidence. The boundaries are blurred, and the lack of consensus only adds to the confusion.
To help my thinking on this I looked again at some articles on cheating over time (going back to 1941!) that I had put in a folder and badly labelled as per usual and selected a few to give me a sense of the what and how as well as the why and to provide a baseline to inform the context around the current assumptions about cheating. Yu et al. (2018) use a long established categorisation of types of cheating with a modification to acknowledge unauthorised digital assistance:
Copying sentences without citation.
Padding a bibliography with unused sources.
Using published materials without attribution.
Accessing exam questions or answers in advance.
Collaborating on homework without permission.
Submitting work done by others.
Giving answers to others during an exam.
Copying from another student in an exam.
Using unauthorised materials in an exam.
The what and how question reveals plenty of expected ways of cheating, especially in exams but it is also noted where teachers / lecturers are surprised by the extent and creativity. Four broad types:
Plagiarism in various forms from self, to peers to deliberate inappropriate practices in citation.
Homework and assignment cheating such as copying work, unauthorised collaboration, or failing to contribute fairly.
Other academic dishonesty such as falsifying bibliographies, influencing grading or contract cheating.
In exams.
The amount of exam based cheating reported should really challenge assumptions about the security of exams at the very least and remind us that they are no panacea whether we see this issue through an ongoing or a chatgpt lens. Stevens and Stevens (1987) in particular share some great pre-internet digital ingenuity and Simpkin and McLeod (2006) show how the internet broadened the scope and potential. These are some of the types reported over time:
Using unauthorised materials.
Obtaining exam information in advance.
Copying from other students.
Providing answers to other students.
Using technology to cheat (using microcassettes, pre-storing data in calculators, mobile phones. Not mentioned but now apparently a phenomenon is use of bone conduction tech in glasses and/ or smart glasses).
Using encoded materials (rolled up pieces of paper for example).
Hiring a surrogate to take an exam.
Changing answers after scoring (this one in Drake,1941)
Collaborating during an exam without permission.
These are the main reasons for cheating across the decades I could identify (from across all sources cited at the end):
Difficulty of the work. When students are on the wrong course (I’m sure we can think of many reasons why this might occur), teaching is inadequate or insufficiently differentiated.
Pressure to succeed. ‘Success’ when seen as the principal goal can subdue the conscience.
Laziness. This is probably top of many academics’ assumptions and it is there in the research but also worth considering what else competes for attention and time and how ‘I can’t be bothered’ may also mask other issues even in self-reporting.
Perception that cheating is widespread. If students feel others are doing it and getting away with it, it increases the cheating.
Low risk of getting caught.
Sense of injustice in systemic approach, structural inequalities both real and perceived can be seen as a valid justification.
External factors such as evident cheating in wider society. A fascinating example of this was suggested to me by an academic who was trained in Soviet dominated Eastern Europe who said cheating was (and remains) a marker of subversion so carries its own respectability)
Lack of understanding of what is allowed and is not- students reporting they have not been taught this and degrees of cheating blurred by some of the other factors here- when does collaboration become collusion?
Cultural influences. Different norms and expectations can create issues and this comes back to my point about individualised (or contextualised) definitions of what is and is not appropriate.
My own experiences, over 30 years, of dealing with plagiarism cases often reveals very powerful, often traumatic, experiences that lead students to act in ways that are perceived as cheating.
For each it’s worth asking yourself:
How much is the responsibility for this on the student and how much on the teacher/ lecturer and / or institution (or even society)?
I suspect that the truly willful, utterly cynical students are the ones least likely to self declare and are least likely to get caught. This furthers my own discomfort about the mechanisms we rely (too heavily?) on to judge integrity too.
This skim through really did make clear to me that cheating and plagiarism are not the simple concepts that many say they are. Also cheating in exams is a much bigger thing than we might imagine. The reasons for cheating are where we need to focus I think. Less so the ‘how’ as that becomes a battleground and further entrenches ‘us and them’ conceptualisations. When designing curricula and assessments the unavoidable truth is we need to do better by moving away from one size fits all approaches, by realising cultural, social and cognitive differences will impact many of the ‘whys’ and hold ourselves to account when we create or exacerbate structural factors that broaden likelihood of cheating.
I am definitely NOT saying give wilful cheaters a free pass but all the work many universities are doing on assessment reform needs to be seen through a much longer lens than the generative AI one. To focus only on that is to lose sight of the wider and longer issue. We DO have the capacity to change things for the better but that also means that many of us will be compelled (in a tense, under threat landscape) to learn more about how to challenge conventions and even invest much more time in programme level, iterative, AI cognisant teaching and assessment practices. Inevitably the conversations will start with the narrow and hyped and immediate manifestations of inappropriate AI use but let’s celebrate this as leverage; as a catalyst. We’d do well, at the very least, to reconsider how we define cheating, why we consider some incredibly common behaviours as cheating (is it collusion or is it collaboration for example or proof reading help from 3rd parties). Beyond that, we should be having serious discussions about augmentation and hybridity in writing: what counts as acceptable support? How does that differ according to context and discipline? It will raise questions about the extent to which writing is the dominant assessment medium, about authenticity in assessment and about the rationale and perceived value of anonymity.
It’s interesting to read how over 80 years ago (Drake, 1941) many of the behaviours we witness today in both students and their teachers have 21st century parallels. Strict disciplinarian responses or ignoring it because ‘they’re only harming themselves’ being common. In other words, the underlying causes were not being addressed. To finish I think this sets out the challenge confronting us well:
“Teachers in general, and college professors in particular, will not be enthusiastic about proposed changes. They are opposed to changes of any sort that may interfere with long- established routines-and examinations are a part of the hoary tradition of the academic past”
(Drake, 1941, p.420)
Drake, C. A. (1941). Why students cheat. Journal of Higher Education, 12(5)
Hutton, P. A. (2006). Understanding student cheating and what educators can do about it. College Teaching, 54(1), 171–176. https://www.jstor.org/stable/27559254
Miles, P., et al. (2022). Why Students Cheat. The Journal of Undergraduate Neuroscience Education (JUNE), 20(2):A150-A160
Rettinger, D. A., & Kramer, Y. (2009). Situational and individual factors associated with academic dishonesty. Research in Higher Education, 50(3), 293-313. https://doi.org/10.1007/s11162-008-9116-5
Stevens, G. E., & Stevens, F. W. (1987). Ethical inclinations of tomorrow’s managers revisited: How and why students cheat. Journal of Education for Business, 63(1), 24-29. https://doi.org/10.1080/08832323.1987.10117269
Yu, H., Glanzer, P. L., Johnson, B. R., Sriram, R., & Moore, B. (2018). Why college students cheat: A conceptual model of five factors. The Review of Higher Education, 41(4), 549-576. https://doi.org/10.1353/rhe.2018.0025
Gallant, T. B., & Drinan, P. (2006). Organizational theory and student cheating: Explanation, responses, and strategies. The Journal of Higher Education, 77(5), 839-860. https://www.jstor.org/stable/3838789
I have been thinking a lot recently about my own and others’ positions in relation to AI in education. I’m reading a lot more from the ‘ResistAI’ lobby and share many persepctives with core arguments. I likewise read a lot from the tech communities and enthusiastic educator groups which often get conflated but are important to distinguish given bloomin’ obvious as well as more subtle agenda and motivation differences (see world domination and profit arguments for example). I see willing adoption, pragmatic adoption, reluctant adoption and a whole bunch of ill-informed adoption/ rejection too. My reality is that staff and students are using AI (of different types) in different ways. Some of this is ground-breaking and exciting, some snag-filled and disappointing, some ill-advised and potentially risky. Exisiting IT infrastrucutre and processes are struggling to keep pace and daily conversations range from ‘I have to show you this- it’s going to change my life! ‘ to ‘I feel like I’m being left behind here’ and a lot more besides.
So it was that this morning I saw a post on LinkedIn (who’d have thought the place where we put our CVs would grow so much as an academic social network?) from Leon Furze who defines his position as ‘sitting on the fence’. I initially I thought ‘yeah that’s me’ but, in fact, I am not actually sitting on the fence at all in this space. I am trying as best I can to navigate a path that can be defined by the broad two word strategy we are trying define and support at my place: Engage Responsibly. Constructive resitance and debate are central but so is engagement with fundamental ideas, technologies, principles and applications. I have for ages been arguing for more nuanced understanding. I very much appreciate evidence and experiential based arguments (counter and pro). The waters are muddied though with, on the one hand, big tech declarations of educational transformation and revolution (we’re always on the cusp, right?) and sceptical generalisations like the one I saw gaining social media traction the other day which went something like:
“Reading is thinking
Writing is thinking
AI is anti-thinking”
If you think that then you are not thinking in my view. Each of those statements must be contextualised and nuanced. This is exactly the kind of meme-level sound bite that sounds good initially but is not what we should be entertaining as a position in academia. Or is it? Below are some adjectives and defintions of the sorts of positions identified by Leon Furze in the collection linked above and by me and research partners in crime Shoshi, Olivia and Navyasara. Which one/s would you pick to define your position? (I am aware that many of these terms are loaded; I’m just interested in the broadest sense where people see themselves, whether they have planted a flag or if they are still looking for a spot as they wander around in the traffic wide eyed).
Cautious: Educators who are cautious might see both the potential benefits and risks of AI. They might be hesitant to fully embrace AI without a thorough understanding of its implications.
Critical: Educators who are critical might take a stance that focusses on one or more of the ethical concerns surrounding AI and its potential negative impacts, such as the risk of AI being used for surveillance or control, or ways in which data is sourced or used.
Open minded: Open minded educators might be willing to explore AI’s possibilities and experiment with its use in education, while remaining aware of potential drawbacks.
Engaged: Engaged educators actively seek to understand AI, its capabilities and its implications for education. They seek to shape the way AI is used in their field.
Resistant: Resistant educators might actively oppose the integration of AI into education due to concerns about its impact on teaching, learning or ethical considerations.
Pragmatic: Pragmatic educators might focus on the practical applications of AI in education, such as using it for administrative tasks or to support personalised learning. They might be less concerned with theoretical debates and more interested in how AI can be used to improve their practice.
Concerned: Educators who are concerned might primarily focus on the potential negative impacts of AI on students and educators. They might worry about issues like data privacy, algorithmic bias, or the deskilling of teachers.
Hopeful: Hopeful educators might see AI as a tool that can enhance education and create new opportunities for students and teachers. They might be excited about AI’s potential to personalise learning, provide feedback and support students with diverse needs.
Sceptical: Sceptical educators might question the claims made about AI’s benefits in education and demand evidence to support its effectiveness. They might be wary of the hype surrounding AI and prefer to wait for more research before adopting it.
Informed: Informed educators would stay up-to-date with the latest developments in AI and its applications in education. They would understand both the potential benefits and risks of AI and be able to make informed decisions about its use.
Fence-sitting: Educators who are fence-sitting recognise the complexity of the issue and see valid arguments on both sides. They may be delaying making a decision until more information is available or a clearer consensus emerges. This aligns with Furze’s own position of being on the fence, acknowledging both the benefits and risks of AI.
Ambivalent: Educators experiencing ambivalence might simultaneously hold positive and negative views about AI. They may, for example, appreciate its potential for personalising learning but be uneasy about its ethical implications. This reflects cognitive dissonance, where conflicting ideas create mental discomfort. Furze’s exploration of both the positive potential of AI and the reasons for resisting it illustrates this tension.
Time-poor: Educators who are time-poor may not have the capacity to fully (or even partially) research and understand the implications of AI, leading to delayed decisions or reliance on simplified viewpoints.
Inexperienced: Inexperienced educators may lack the background knowledge to confidently assess the potential benefits and risks of AI in education, contributing to hesitation or reliance on the opinions of others.
I have been interested recently in the ways in which AI is being integrated into healthcare as part of my personal goal to widen my understanding and broaden my own definition of AI. I’m seeing increasing need to do this as part of growing awareness and literacy as well as a need to show that AI is impacting curricula well beyond the ongoing kerfuffle around generative AI and assessment integrity. I was recommended this panel by Professor Dan Nicolau Jr who chaired this session at the recent event which looked at the many barriers to advances in a context where early detection, monitoring, business models and data availability impact the ways in which we do medicine and advance it in a world where ageing populations present an existential threat to global healthcare systems. It struck me when I watched this how much the potentials and barriers expressed here will likely be mirrored in other disciplines. Medicine does seem to be an effective bellweather though.
Some of the issues that stood out:
Data availability and validity: Just as healthcare AI can produce skewed results from over-represented organisms in protein design, we see similar issues of data bias emerging across AI applications. The challenges around electronic health records – inconsistent, incomplete and error-prone – mirror concerns about data quality in other domains.
Business models and willingness/ ability to use what is available: The difficulty in monetising preventative AI applications in medicine, for example, reflects broader questions about how we value different types of AI innovation. Similarly, the need to shift mindsets from reactive to proactive approaches in healthcare has parallels with cultural change required for effective AI adoption elsewhere. The comments from the panel about human propensities NOT to use devices or take medicines that will help them are quite shocking but still somehow unsurprising. Cracking that, according to the panel, would increase life expectancy more than finding a cure for cancer.
The regulatory landscape: The NHS’s procurement processes, which can stifle AI innovation, demonstrate how existing institutional frameworks may need significant adaptation. This raises important questions about how we balance innovation with appropriate oversight – something all sectors grappling with AI must address.
For me, healthcare exemplifies the complex relationship between technical capability and human behaviour. The adoption issue is obviously one that has parallels with willingness/ openness to using novel technologies, even where they can be shown to make life better or easier. The panel’s observations about patient compliance mirror wider challenges around user adoption and engagement with AI systems. We cannot separate the technology from the human context in which it operates.
This is a swift intro to character AI *(note 1)- a tool that is available to use for free currently (on a freemium model). My daughter showed it to me some months ago. It appears as a novelty app but is used (as I understand it) beyond entertainment for creative activity, gaming, role playing and even emotional support. For me it is the potential to test ideas that many have about bot potential for learning that is most interesting. By shifting focus away from ‘generating essays’ it is possible to see the appeal of natural language exchanges to augment learning in a novel medium. While I can think of dozens of use cases based on the way I currently (for example) use YouTube to help me to learn how to unblock a washing machine I imagine that is a continuum that goes all the way up to teacher replacement.*(note 2) Character AI is built on large language model, employs ‘reinforcement’ (learning as coversations continue) and provides an easy to learn interface (basically typing stuff in boxes) that allows you to ground the bot with ease in a wysiwyg interface.
As I see it, it offers three significant modifications in the default interface to standard (free) LLMs. 1. You can create characters and define their knowledge and ‘personality’ traits by having space to ground the bot behaviour through customisation. 2. You can have voice exchanges by ‘calling’ the character. 3. Most importantly, it shifts the technology back to interaction and away from lengthy generation (though they can still go on a bit if you don’t bake succinctness in!) What interests me most is the potential to use tools like this to augment learning, add some novelty and provide reinforcement opportunity through text or voice based exchanges. I have experimented with creating some academic architypes for my students to converse with. This one is a compassionate pedagogue, this one is keen on AI for teaching and learning, this one a real AI sceptic, this one deeply worried about academic integrity. They each have a back story, defined university role and expertise. I tried to get people to test arguments and counter arguments and to work through difficult academic encounters. It’s had mixed reviews so far: Some love it; some REALLY do not like it at all!
How do/ could you use a tool like this?
Note 1. This video in no way connotes promotion or recommendation (by me or by my employer) of this software. Never upload data you are not comfortable sharing and never upload your own or other’s personal data.
Note 2: I am not a proponent of this! There may be people who think this is the panacea to chronic educational underfunding though so beware.
In the fast evolving landscape of AI tools, two recent releases have really caught my attention: Google’s NotebookLM and the advanced conversational features in ChatGPT. Both offer intriguing possibilities for how we might interact with AI in more natural, fluid ways.
NotebookLM, still in its experimental stage and free to use, is well worth exploring- as one of my King’s colleagues pointed out recently: it’s about time Google did something impressive in this space! Its standout feature is the ability to generate surprisingly natural-sounding ‘auto podcasts’. I’ve been particularly struck by how the AI voice avatars exchange and overlap in their speech patterns, mimicking the cadence of real conversation. This authenticity is both impressive and slightly unsettling and at least two colleagues thought they were listing to human exchanges.
I tested this feature with three distinct topics:
Language learning in the age of AI (based on three online articles):
A rather flattering exchange about my blog posts (created in fact by my former colleague Gerhard Kristandl – I’m not that egotistical):
A summary of King’s generative AI guidance:
The results were remarkably coherent and engaging. Beyond this, NotebookLM offers other useful features such as the ability to upload multiple file formats, synthesise high-level summaries, and generate questions to help interrogate the material. Perhaps most usefully, it visually represents the sources of information cited in response to your queries, making the retrieval-augmented generation process transparent.
Meanwhile, ChatGPT’s latest update advance voice feature (not available in EU, by the way) has addressed previous latency issues, resulting in a much more realistic exchange. To test this, I engaged in a brief conversation, asking it to switch accents mid-dialogue. The fluidity of the interaction was notable, feeling much closer to a natural conversation than previous iterations. Watch here:
What struck me during this exchange was how easily I slipped into treating the AI as a sentient being. At one point, I found myself saying “thank you”, while at another I felt a bit bad when I abruptly interrupted. This tendency to anthropomorphise these tools is deeply ingrained and hard to avoid, especially as the interactions become more natural. It raises interesting questions about how we relate to AI and whether this human-like interaction is beneficial or potentially problematic.
These developments challenge our conventions around writing and authorship. As these tools become more sophisticated, the line between human and AI-generated content blurs further. What constitutes a ‘valid’ tool for authorship in this new landscape? How do we navigate the ethical implications of using AI in this way?
What are your thoughts on these developments? How might you see yourself using tools like NotebookLM or the advanced ChatGPT in your work?
Read my post below or listen to AI me read it. Have to say, I sound very well spoken in this video. To my ears doesn’t sound much like me. For those that know me: what do you think?
As we attempt to navigate uncharted (as well as expanding and changing) landscapes of artificial intelligence in higher education, it makes sense to reflect on our approaches and understanding. We’ve done ‘headless chicken’ mode; we’ve been in reactive mode. Maybe we can start to take control of the narratives; even if what is ahead of us is disrupting, fast-moving and fraught with tensions. Here are six perspectives from me that I believe will help us move beyond the hype and get on with the engagement that is increasingly pressing but, thus far, inconsistent at best.
1. AI means whatever people think it means
In educational circles, when we discuss AI, we’re primarily referring to generative tools like ChatGPT, DALL-E, or Copilot. While computer scientists might argue- with a ton of justification- this is a narrow definition, it’s the reality of how most educators and students understand and engage with AI. We mustn’t get bogged down in semantics; instead, we should focus on the practical implications of these tools in our teaching and learning environments whilst taking time to widen some of those definitions, especially when talking with students. Interrogating what we mean when we say ‘AI’ is a great starting point for these discussions in fact.
2. AI challenges our identities as educators
The rapid evolution of AI is forcing us to reconsider our roles as educators. Whether you buy into the traditional framing of higher education this way or not, we’re no longer the sole gatekeepers of knowledge, dispensing wisdom from the lectern. However much we might want to advocate for notions of co-creation or discovery learning, the lecturer/ teacher as expert is a key component of many of our teacher professional identities. Instead, we need to acknowledge that we’re all navigating this new landscape together – staff and students alike. This shift requires humility and a willingness to learn alongside our students. The alternatives: Fake it until you make it? Bury your head? Neither are viable or sustainable. Likewise, this is not something that is ‘someone else’s job’. HE is being menaced from many corners and workload is one of the many pressures- but I don’t see a beneficial path that does not necessitate engagement. If I’m right then something needs to give. Or be made less burdensome.
3. Engage, not embrace
I’m not really a hugger, tbh. My family? Yes. A cute puppy? Probably. Friends? Awkwardly at best. A disruptive tech? Of course not. While some advocate for ’embracing’ AI, I prefer the term ‘engage’. We needn’t love these technologies or accept them unquestioningly, but we do need to interact with them critically and thoughtfully. Rejection or outright banning is increasingly unsupportable, despite the many oft-cited issues. The sooner we at least entertain the possibilities that some of our assumptions about the nature of writing and what constitutes cheating and how we best judge achievement may need review the better.
4. AI-proofing is a fool’s errand
Attempts to create ‘AI-proof’ assessments or to reliably detect AI-generated content are likely to be futile. The pace of technological advancement means that any barriers we create will swiftly be overcome. Many have written on the unreliability and inherent biases of detection tools and the promotion of flawed proctoring and surveillance tools only deepens the trust divide between staff and students that is already strained to its limit. Instead, we should focus on developing better, more authentic forms of assessment that prioritise critical thinking and application of knowledge. A lot of people have said this already, so we need to build a bank of practical, meaningful approaches, draw on the (extensive) existing scholarship and, in so doing, find ways to better share things that address some of the concerns that are not: ‘Eek, everyone do exams again!’
5. We need dedicated AI champions and leadership
To effectively integrate AI into our educational practices, we need people at all levels of our institutions who can take responsibility for guiding innovations in assessment and addressing colleagues’ questions. This requires significant time allocation and can’t be achieved through goodwill alone. Local level leadership and engagement (again with dedicated time and resource) is needed to complement central policy and guidance. This is especially true of multi-faculty institutions like my own. There’s only so much you can generalise. The problem of course is that whilst local agency is imperative, too many people do not yet have enough understanding to make fully informed decisions.
6. Find a personal use for AI
To truly understand the potential and limitations of AI, it’s valuable to find ways to develop understanding with personal engagement – one way to do this is to incorporate it into your own workflows. Whether it’s using AI to summarise meeting or supervision notes, create thumbnails for videos, or transform lecture notes into coherent summaries, personal engagement with these tools can help demystify them and reveal practical benefits for yourself and for your students. My current focus is on how generative AI can open doors for neurodivergent students and those with disabilities or, in fact, any student marginalised by the structures and systems that are slow to change and privilege the few.
*That’s supposed to read AI3 but the title font refuses to allow superscript!
Yesterday I was delighted to keynote at the Universities at Medway annual teaching and learning conference. It’s a really interesting collaboration of three universities: University of Greenwich, University of Kent and Canterbury Christchurch University. Based at the Chatham campus in Medway you can’t help but notice the history the moment you enter the campus. Given that I’d worked at Greenwich for five years I was familiar with the campus but, as was always the case when I went there during my time at Greenwich, I experienced a moment of awe when seeing the campus buildings again. It’s actually part of the Chatham Dockyard World Heritage site and features the remarkable Drill Hall library. The reason I’m banging on about history is because such an environment really underscores for me some of those things that are emblematic of higher education in the United Kingdom (especially for those that don’t work or study in it!)
It has echoes of cultural shorthands and memes of university life that remain popular in representations of campus life and study. It’s definitely a bit out of date (and overtly UK centric) like a lot of my cultural references, but it made me think of all the murders in the Oxford set crime drama ‘Morse’. The campus locations fossilised for a generation the idea of ornate buildings, musty libraries and deranged academics. Most universities of course don’t look like that and by and large academics tend not to be too deranged. Nevertheless we do spend a lot of time talking about the need for change and transformation whilst merrily doing things the way we’ve done them for decades if not hundreds of years. Some might call that deranged behaviour. And that, in essence, was the core argument of my keynote: For too long we have twiddled around the edges but there will be no better opportunity than now with machine-assisted leverage to do the things that give the lie to the idea that universities are seats of innovation and dynamism. Despite decades of research that have helped define broad principles for effective teaching, learning, assessment and feedback we default to lecture – seminar and essay – report – exam across large swathes of programmes. We privilege writing as the principle mechanism of evidencing learning. We think we know what learning looks like, what good writing is, what plagiarism and cheating are but a couple of quick scenarios to a room full of academics invariably reveal lack of consensus and a mass of tacit, hidden and sometimes very privileged understandings of those concepts.
Employing an undoubtedly questionable metaphor and unashamedly dated (1984) concept of ‘crossing the streams’ from the original Ghostbusters film, I argued that there are several parallels to the situation the citizens of New York first found themselves in way back when and not least the academics (initially mocked and defunded) who confront the paranormal manifestations in their Ghostbusters guises. First are the appearances of a trickle of ghosts and demons followed by a veritable deluge. Witness ChatGPTs release, the unprecedented sign ups and the ensuing 18 months wherein everything now has AI (even my toothbrush). There’s an AI for That has logged 12,982 AIs to date to give an indication of that scale (I need to watch the film again to get an estimate on number of ghosts). Anyway, early in the film we learn that a Ghost catching device called a ‘Proton Pack’ emits energy streams but:
“The important thing to remember is that you must never under any circumstances, cross the streams.” (Dr Egon Spengler)
Inevitably, of course, the resolution to the escalating crisis is the necessity of crossing the streams to defeat and banish the ghosts and demons. I don’t think that generative AI is something that could or should be defeated and I definitely do not think that an arms race of detection and policing is the way forward either. But I do think we need to cross the streams of the three AIs: Artificial Intelligence; Academic Integrity and Assessment Innovation to help realise the long-needed changes.
Artificial Intelligence represents the catalyst not the reason for needing dramatic change.
Academic Integrity as a goal is fine but too often connotes protected knowledge, archaic practices, inflexible standards and a resistance to evolution.
Assessment innovation is the place where we can, through common language and understanding, address the concerns of perhaps more traditional or conservative voices about perceived robustness of assessments in a world where generative AI exists and is increasingly integrated into familiar tools along with what might be seen as more progressive voices who, well before ChatGPT, were arguing for more authentic, dialogic, process-focussed and, dare I say it, de-anonymised and humanly connected assessments.
Here is our opportunity. Crossing the streams may be the only way we mitigate a drift to obsolescence! MY concluding slide showed a (definitely NOT called Casper) friendly ghost which, I hope, connoted the idea that what we fear is the unknown but as we come to know it we find ways to shift from engagement (sometimes aggressively) to understanding and perhaps even an ‘embrace’ as many who talk of AI encourage us to do.
Incidentally, I asked the Captain (in my custom bot ‘Teaching Trek: Captain’s Counsel’) a question about change and he came up with a similar metaphor:
“Blow Up the Enterprise: Sometimes, radical changes are necessary. I had to destroy the Enterprise to save my crew in “Star Trek III: The Search for Spock.” Academics should learn when to abandon a failing strategy and embrace new approaches, even if it means starting over.”
In a way I think I’d have had an easier time if I’d stuck with Star Trek metaphors. I was gratified to note that ‘The Search for Spock’ was also released in 1984. An auspicious year for dated cultural references from humans and bots alike.
—————–
Thanks:
The conference itself was great and I am grateful to Chloe, Emma, Julie and the team for orgnaising it and inviting me.
Earlier in the day I was inspired by presentations by colleagues from the three universities: Emma, Jimmy, Nicole, Stuart and Laura. The student panel was great too- started strongly with a rejection of the characterisation of students as idle and disintersted and carried on forcefully from there! And special thanks too to David Bedford (who I first worked with something like 10 years ago) who uses an analytical framework of his own devising called ‘BREAD’ as an aid to informing critical information literacy. His session adapted the framework for AI interactions and it prompted a question which led, over lunch, to me producing a (rough and ready) custom GPT based on it.
I should also acknowledge the works I referred to: 1. Sarah Eaton whose work on the 6 tenets of post-plagiarism I heartily recommended and to 2. Cath Ellis and Kane Murdoch* for their ‘enforcement pyramid’ which also works well as one of the vehicles that will help us navigate our way from the old to the new.
*Recommendation of this text does not in any way connote acceptance of Kane’s poor choice when it comes to football team preference.
HEducationist 