Seems like proctored, handwritten essay exams are the best way to deal with it all.
Yeah but then someone has to actually read it to grade it
Then use locked down computers that doesn’t do anything, but display the exam and student can select multiple choice and type where prompted?
Its all solutions that existed before AI. Nothing has really changed over the past decade when it comes to not allowing or restricting electronics during exams other than schools forgetting they can do that.
Students looking up answers over the internet or getting text messages were just as much of a problem as AI if exams allowed students to bring those devices and use it freely. Even graphing calculator usage was restricted by smarter teachers aware of its note storing and program capabilities.
remember these?
Yeah, it’s a made up problem. In my experience, generated by two factors: universities wanting to increase the number of tests for each subject, and wanting students to be experienced with using computers. Thus many tests become at home, with the computer, where cheating is soooooo easy.
Yeah, cheating with electronics is nothing new. Laptop and phones and internet has existed long before AI, and people could get outside help through messages to have people feed them answers.
Yet, schools dealt with it by either doing in class paper and pencil exams, or digital in class exams on locked down computers with no internet access and no desktop access with software like examsoft.
I’ve had even math teachers ban graphing calculators or provide use of in class graphing calculators well aware of notes that could be typed into them and programs made to solve answers before hand.
So all these current issues seem to be the result of either schools moving to take home, or letting people use personal electronics with full internet and file access during exams. Its like teachers lost the ability to say no to what is and isn’t allowed during exams.
In class graphing calculators would be good.
Our teachers instead manually wiped our ti-83+ before the exam.
Fortunately for us, we had a program that emulated the system menus, so the teacher could think they were resetting it without actually doing it.
Some people used it to cheat but most of us just didn’t want to lose all our games.
Yeah, I think archiving the programs would lead to surviving a reset.
They didn’t. There was a type of reset that restores the calculator to factory settings iirc
Looks like that type was ‘All Memory’
https://education.ti.com/en/customer-support/knowledge-base/ti-83-84-plus-family/product-usage/34871
I can think of a couple of rules that could prevent a lot of the more blatant cheating:
-
All computer generated coursework must have some form of change tracking enabled, thus the assessor can see the way the essay was written.
-
All computer generated coursework must come with an accompanying oral assessment where the student has to discuss their work and research methodology.
The former would make it difficult to copy directly from ChatGPT, as the lack of any editing would leave an obvious fingerprint
… And passing the latter would require a level of understanding that would be difficult to replicate if you hadn’t done the research yourself.
Passing the latter is basically the same step as a tradesperson needs to complete to show they know what they are doing. It is completely reasonable and would also catch plagiarism and paying some other person to do the work in addition to ai.
-
The gold standard would be to make each student do an oral defense of their work. Hand-written exams are good too. Both of these approaches are time-consuming to mark, though.
Assessment design can require students to perform analysis of rare and obscure source material, so that there are no published materials for AI to draw on (which can make it struggle). Similarly, asking for a good bibliography that lists the sources used in the text and formats it correctly would show up the hallucinatory tendencies of AI, I think — though I haven’t tried the latest models.
Agreed 100%.
Every exam should have some oral component. Doesn’t need to be the whole of one’s grade, but 10% sounds about right: significant enough to be impactful, but not so significant that people with severe anxiety can’t otherwise pass.
My biggest fear was this. I hated getting up information of a class
Same, but it motivated me to do the work, and it’s a skill that crosses over into work life significantly.
Doesn’t scale.
It works great. That’s basically how it’s done in aviation, flight instruction is inherently 1 on 1, so I’d assign my students to do some reading and then it was pretty obvious if they’d done that the next flight based on who was asking questions. The ones that did the reading will come in asking me to clarify something, if they say “Yeah I read it” and I have to do the clarifying they didn’t read it. Oral and practical tests are the standard, show me you can fly the plane.
To make that work, most professional pilots have to be flight instructors at some point. It’s almost certainly the first flying job you’ll get after earning a commercial certificate, and thus spend at least some time doing that job. You’re not going to get that in most college classes, so where are you going to get the labor necessarily to give an oral exam to every student in every class?
Yes — I think if I were to teach a large undergraduate class tomorrow I would go with a combination of handwritten exams and, for homework assignments, critical analyses of the most obscure examples of whatever is being studied possible so that AI has no idea what you’re even on about (in my field, ask it to analyse music; it’s hopeless). Multiple choice is fine for stuff that can be rote learned.
As someone who is terrible at project work but great at tests, maybe now is finally my time to get a degree. Because tests seem the only thing you can’t cheat on now.
An interesting gotcha that I don’t see universities actually addressing on any of this…
English-as-a-Second-Language students are generally being given a pass to use AI freely. The thought is that proper grammar and writing aren’t what’s being graded for many papers.
The problem is that they can just let AI write the paper and say “I only used AI for grammar”.
Reap what you’ve sown, assholes.
What are you on about? This is about how universities have a hard time managing AI use and misuse by students. Have they sown those AI tools?
The essence of an academic paper is that a student shows that they have learned all the goals that were set for that course or courses. If a student lets tgere paper be generated by AI, they can not prove that they have learned those goals. That is why universitues have (or should have) strict AI guidelines, so that students know which kinds of use are allowed and which are not.
This is a hard nut to crack. Of course you will see overfitting and underfitting until someone somewhere fimds a universal approach. I personally think AI-detection tools are shit, but you cannot fault universities for trying it. My university handles AI fraud manually, with a committe. That is very labor intensive. We only have a couple hundred students, but if you have thousands, it quickly becomes impossible.
Can you elaborate?
Universities started pushing AI as a learning tool. They pay for subscription AI services for enrolled students. So I’m not sure this isn’t them doing this to themselves.
These places that went all in on hyped up “AI” and dystopian surveillance can take their lumps. They’re “realizing” what they’ve been told from day one.
Happy cake day!