Episode

Episode | Quick Take

In episode 149 of The A&P Professor podcast, host Kevin Patton chats with Greg Crowther and Ben Wiggins about their work with exams in the anatomy and physiology (A&P) course. They discuss the importance of exams in assessing student learning and the need for exams to be more connected to course objectives. They also mention the challenges of designing exams that are fair and inclusive for all students. Crowther and Wiggins are conducting a survey on A&P exams and encourage listeners to participate to contribute to the understanding of exam practices in the A&P community. The survey can be accessed at tinyurl.com/stemexamsurvey.

0:00:00 | Introduction
0:00:48 | Introducing Ben & Greg
0:04:17 | The Most Important Thing
0:22:32 | Murray Jensen, HAPS Hero
0:23:46 | Our Motto: Be Prepared
0:41:18 | What’s on TAPP at The Corner Pub
0:42:45 | The Next Big Leap: What Is It?
1:00:50 | Staying Connected

Need help accessing resources locked behind a paywall?
Check out this advice from Episode 32 to get what you need!

Episode | Transcript

The A&P Professor podcast (TAPP radio) episodes are made for listening, not reading. This transcript is provided for your convenience, but hey, it’s just not possible to capture the emphasis and dramatic delivery of the audio version. Or the cool theme music.  Or laughs and snorts. And because it’s generated by a combo of AI robot and human transcription, it may not be exactly right. So I strongly recommend listening by clicking the audio player provided or the captioned audiogram.

Introduction

Kevin Patton (00:00:00):
Exactly 200 years ago, a book of Percy Bysshe Shelley’s poems was published posthumously. It contains this line, “The more we study, the more we discover our ignorance.”

Aileen Park (00:00:16):
Welcome to The A&P Professor, a few minutes to focus on teaching human anatomy and physiology with a veteran educator and teaching mentor, your host, Kevin Patton.

Kevin Patton (00:00:26):
In episode 149, I chat with Ben Wiggins and Greg Crowther about their work with exams in the A&P course.

Introducing Ben & Greg

Kevin Patton (00:00:37):
I recently had an interesting and enjoyable conversation with my friends, Greg Crowther and Ben Wiggins, and we were talking about the exam in the A&P course, and I’m going to present that whole conversation to you in this episode. But before we do that, I want to introduce you to them.

(00:01:08):
Greg Crowther is a tenured faculty member at Everett Community College that’s just north of Seattle and just north of where Ben is. And Greg has been there since 2018. He’s got a PhD in physiology and biophysics as well as a master’s degree in science education. And Greg has achieved some success in ultra marathoning. Yeah, you heard that right, ultra marathoning. Between 2005 and 2010, he represented the United States three times at the 100K World Cup. Greg is married to a biostatistician and they have three boys, ages 17, seven and five.

(00:01:52):
And before focusing on assessment issues, Greg’s scholarly work focused on educational science music. If you’re a longtime listener, you’ve heard some of his music from the early days of this podcast. Every once in a while we’d bring in some of Greg’s STEM themed music. And Greg has been featured in a previous episode discussing test question templates, or TQTs. That was back in episode 70, but he has also called in a few times on the podcast hotline.

(00:02:26):
So I don’t know, which is braver doing these ultra-marathons or calling in the podcast hotline. And the reason why I’m not sure is there are a bunch of people that do ultra marathoning. Now, not a huge number, but there are a number. That’s not so with people calling into the podcast hotline. There are a very small group of people that call into the podcast hotline and you might want to join them and join the bravest of the brave who do that.

(00:02:59):
Now Ben Wiggins, who’s also joining us for this conversation, he teaches A&P 1 and A&P 2 at Shoreline Community College, and he’s a tenure track professor in the biology department there. He’s also got a master’s degree in biology and a PhD in science education. And he’s worked with coaches and athletes across the United States and in Russia, China, the Philippines, Colombia, and well, just all over the place. Keep that thought in mind because his experience with coaches is something that’s going to come up in this conversation.

(00:03:38):
Ben lives in Seattle with his partner who’s a fantastic middle school teacher by all accounts, and they have two kids, age nine and five who reportedly act like they’re in middle school. Ben does research focusing on early career training for great professors and ambitious science teaching. Perhaps most importantly, he is an expert in Minesweeper. His best Minesweeper time is 77 seconds. So let’s go ahead and join Greg and Ben regarding exams in the A&P course.

The Most Important Thing

Kevin Patton (00:04:17):
Well, I’m here with Greg and Ben who I just introduced to you, and I’m really looking forward to our conversation because these guys are into all kinds of things that really interest me. This has a goal to it, a practical goal of them stopping by. This isn’t just a random thing where I called them up and said, “Hey, Greg, hey Ben, you want to come and talk about your latest stuff?” They actually approached me. It’s sort of like on those talk shows where some celebrity will go to a talk show and they act as if they just happened to be available, but really they’re promoting their movie or their book or whatever it is.

(00:04:57):
And these guys have a movie, no, not a movie. They have a survey that they’d like you to take. It’s a survey about exams. I just took the survey myself yesterday. I always hesitate about taking surveys because sometimes you start a survey and then you realize 20 minutes into it, there’s probably going to be another hour of questions and you just didn’t plan for that. But this survey is not like that. It’s a short survey.

(00:05:22):
And as I was telling them before we started recording, it’s an interesting survey because it gets you thinking about what you do in exams. So it’s mainly just checking off things like, “Yeah, I do this. No, I don’t do that. Yes. No. Yes. No.” And then at the end there’s opportunity to expand on it and explain, but it’s not required. So it’s an easy thing and I think it’s really going to help them in their work because they’re really taking close work at exams. So let’s start off with that is why are you doing this survey?

Greg Crowther (00:05:55):
Go ahead, Ben.

Ben Wiggins (00:05:59):
Thanks, Kevin. It’s so good to be here and be talking about this because there are so many important parts of teaching and I think we can’t think of anything more important than how we ask our students to show us what they’ve learned. Now that’s not always exams, and for some classes that’s a small percentage of the points. Some classes, that’s a huge percentage of the points. But the college, especially the college science exam, is an experience that translates far beyond the small time that people spend on it.

(00:06:33):
If you put up a picture right now on the internet for anyone in our country or society, and it’s a person at a desk with a piece of paper, a pen and looking stressed, they will immediately start talking about oh, that’s probably an exam. That’s probably in college depending on the person that they’re seeing. Oh, that’s probably a really difficult class. And it immediately telescopes into, well, that’s how they get into their career. Well, that really matters. That determines who goes on or not.

(00:07:02):
It’s this massively disproportionate impact on society, on science, on healthcare, just from this activity that many of us do largely without long training in how to do it. I never had training in how to write exams when I was getting started, and I think a lot of people are the same.

(00:07:22):
So I feel like exams are at this point where they’re so prevalent, they’re so massive in how they impact students, and yet, they’re so under-researched. We know so little about what people are actually doing in their classrooms, and this survey is a way to start opening that field in what we hope are some new and interesting ways.

Kevin Patton (00:07:46):
Exams are something that I think a lot of us instructors don’t like to think about. They’re stressful for instructors in many ways too. I’m curious. I was already familiar with at least some of what both of you have been doing in terms of teaching and things related to pedagogy and so on. Greg, you’ve been on the podcast before. Actually I was looking, it was four years ago. It seems like yesterday. You were talking about TQTs, test question templates. So that kind of gets back to exams too. Ben, I’ve seen some of your blog articles and things like that.

(00:08:20):
And so I know you guys have been working on related issues for a long time and in very engaged ways, but it seemed like you guys were on, I don’t want to say two different tracks intellectually or topic wise, but you didn’t seem to be collaborating at least for a while directly together. So how did that happen? How did you two connect up to join forces and look at some of these issues?

Greg Crowther (00:08:46):
Yes, that’s my cue. Thank you, Kevin, for that lead in. So Ben and I both were currently professors at community colleges, Ben at Shoreline, and me at Everett, but we both used to be affiliated with the University of Washington, specifically the Seattle campus, the main campus. And so I would say Ben and I were casual colleagues for a few years, knew of each other. Ben helped introduce me to the jigsaw technique, but we weren’t close collaborators until 2019 when somehow I stumbled upon this blog post that Ben had written about his public exam system, which I knew nothing about until I read this blog post. And it was super long. It was more of an article than a post.

(00:09:40):
To my great surprise and interest, Ben had been struggling mightily with all of the problems that we face in writing and grading exams for years. And in this post he laid out five distinct problems he had, language difficulties, lower level thinking, unproductive stress, having students feeling disenfranchised by their test scores, and real world constraints on exam development. He said, “After a lot of work, I’ve figured out how to solve all five of these problems through reading other people’s insights and my own approaches.” And so I thought it was absolutely fantastic how he put it together.

(00:10:22):
And as I tell the story, it’s only a slight embellishment to say that I raced down to his office. I was like, “Ben, this is brilliant. I want to talk about this more with you and figure out how I can do some version of this in my classroom.” I think Ben was flattered by the attention. I certainly hope he was, yet I didn’t want to do his exact thing.

(00:10:43):
I had some of my own ideas brewing, and so I enlisted him to help me develop my own version of this, which would ultimately become test question templates or TQTs. But it really all started with Ben writing this very, very thoughtful, very careful analysis of, here are all the things I hate about tests, and here’s how I am finally making those better.

Ben Wiggins (00:11:09):
That was a super nice way of saying it all, Greg. I just want to add in that I feel like the two different ways that we’ve been coming at this problem, I feel like both have really important things to add to the point where when I switched institutions and I was looking for a way to adjust into the community college environment, which has unique challenges and unique students, I took up TQTs as my dominant way of testing right now, partially because I wanted to learn more about, but it works really, really well.

(00:11:40):
So between our two styles, I think that there’s, well, hopefully things that other people can, just like we did, steal the good stuff from other people, and if we can help share that, that’s just awesome.

Kevin Patton (00:11:52):
Each of you started with different aspects of that test experience, that exam experience that we all love to hate and are looking for ways to make them less hateful for both instructors and for the students, which is a noble goal that a lot of us don’t want to tackle because it’s like, “Yeah, I see how big that mountain is. Nah, nevermind. I’ll go round. Or I’ll take the helicopter up to the top instead. I want to avoid that.”

(00:12:26):
But you guys, each in your own way, just dove right into it and then found a way to look at what each other were doing and use that in your own work and it’s evolving. Where are we now with things? I see it as the TQT thing, for example, test question template, it’s a method. Greg, do you have a short capsule, a short two-sentence way of describing what that is and how it works?

Greg Crowther (00:12:53):
I’m still working on my elevator speech, but the one short version is it’s a way of very, very directly and explicitly linking our learning outcomes or learning objectives to specific ways that we will assess achievement of those outcomes or objectives. So it’s just we all have learning outcomes. We all have practice questions and real questions, but just making those connections extra clear and extra direct.

Kevin Patton (00:13:25):
Okay, and you do that by using a template perspective or method, right?

Greg Crowther (00:13:30):
Mm-hmm.

Kevin Patton (00:13:30):
Okay.

Greg Crowther (00:13:31):
Yeah. Another little add-on would be instead of treating every possible exam question as unique TQTs, try to directly show students the larger patterns of this is a type of question that you can now solve, and here are a couple of specific examples of this type of question. And your practice on these examples will help you solve future examples that look different in the details, but are broadly similar.

Kevin Patton (00:14:01):
But follow the same pattern of questioning?

Greg Crowther (00:14:04):
Exactly.

Kevin Patton (00:14:05):
Yeah, okay. And Ben, your public exams, I think you did already summarize what that is, but do you have an elevator speech you can share with us?

Ben Wiggins (00:14:16):
Yeah, I’ve tried different variations of this many times. I spent an embarrassingly large part of my life playing basketball. As important as those games were to me, the one thing that never happened to me is I never walked out onto the court for starting lineups and saw that the hoop was 11 feet high or seven feet high, it was always 10. And in these much more important challenges that we give to students, they open up the paper and often have no idea what’s coming.

(00:14:47):
Public exams are really a way to say, “Look, the hoop is going to be 10 feet high. Let’s give you as much information about the challenge without revealing the actual questions, like the final few percent. Let’s show you what the court is going to be like so you can practice for it, so you can prepare for it, so you can think creatively about the kinds of challenges.” And we can ask higher conceptual questions that avoid simple ChatGPT cheating and I think make us all happier about what we’re teaching at the end of the day.

Kevin Patton (00:15:17):
Okay. Yeah, that makes sense. Basically, what’s your next step? I know that you had shared with me a paper that you were working on that’s not in publication yet, but you’re working on that. You described it, Greg, as a mini review. I saw somewhere in there that you self-referred to it as an essay. So it’s not really a research report, it’s looking at the literature and your own experience and then framing an idea around it, having to do, well, with justice basically. The tentative title is Redistributing Academic Power by Reforming Unjust Exam Practices.

(00:16:00):
When I looked at the title, man, that really caught my attention. It’s like, “Unjust exam practices? I’m not unjust. I try to be just. How could that be?” And then I start reading it and thinking, “Oh my gosh, I’m unjust.” And not intentionally, and I don’t know, I’d like to think it’s not really coming from inside me. But I’m using strategies that have been handed to me or I’ve learned, or maybe I’ve even developed on my own that if I really examine them, they really don’t treat everyone fairly.

(00:16:41):
That’s very challenging to me, but challenging in a good way. It’s like, oh gosh, here are things I’m doing to make this, Ben, as you said, a very important, incredibly important thing, an exam, I’m putting up barriers when I really don’t want to put up barriers. I just want to know how much my students know. I don’t want to know how well they’re playing some game. So can you guys talk more about that? Where are you headed with this?

Ben Wiggins (00:17:12):
Greg, do you want to start? Do you want me to?

Greg Crowther (00:17:15):
Yeah, let me say a little bit and then I’ll hand it off to you. On this secret essay that is not yet published, but hopefully will be someday, I want to make a bit of a strategic comment about that, which is Ben and I are working very hard on assessment issues and trying to make these horrible things better for everyone. But I think one aspect that is not always obvious to people when we talk about this stuff and talk about what we’re doing is there’s a real equity and inclusiveness lens to all of this.

(00:17:52):
There’s a lot of really great, important DEI work going on right now concerning things like sexism and racism and ageism and so forth. And this is what we see as another component of that, which is exams, which are intended to be fair and objective, often assume certain implicit knowledge that students have of the system. And so we’re trying to break down some of those barriers and make our exams more equitable for the benefit of everybody, but especially for those students who might historically have been marginalized due to income, race, sex, gender, et cetera, et cetera. Ben, please take the pass and put it through the hoop from here.

(00:18:47):
It’s funny we’re on a basketball analogy at the moment because Ben’s primary sport actually in recent decades has been ultimate frisbee. And I think a lot of his teaching is informed by his experiences as an ultimate frisbee coach, which to me is very powerful because as a coach, you’re constantly giving feedback that’s like, “That wasn’t quite right, but you did this okay, and now try this.”

(00:19:15):
Which is I think a lot of what we want to do with our students is identify their limitations and then coach them on how to do it better and give them more opportunities. So ideally our exams will be like that, you kind of screwed this up, but there are more chances to come for the next time. Think about this, practice more on this. And so I think that coaching lens is really, really helpful.

Ben Wiggins (00:19:42):
Yeah, I think that’s well said. I mean, I don’t do anything in my classes that I didn’t steal from better teachers and professors before me. And I think that if you have classes where students are getting motivated and you have classes where students are going into careers that they’re looking for, if you have classes where students are excited about talking about the material, you must be doing something right with your teaching, which means because exams are such a big part of teaching, if you use exams, then you must be doing something right on your exams.

(00:20:12):
And so we’re hoping in this essay, paper, mind dump to get people to put exams up on stage as this is a real part of teaching. It’s not something we do after our teaching is done. It’s not this separate extractive process that we all have to do in the same way that we learned it. It’s the way that you assess your students is part and parcel with the other things that you have already done for teaching justice.

(00:20:42):
So some people have taken up active learning in the last decade and a half as a, hey, we are going to do what second grade teachers all did 50 years ago, and we are going to make our classes more just. And that’s an incredible step for the people for whom that is appropriate. Assessment exams are kind of lagging behind that, where first we hit some aspects of teaching. We’ve started to address some of the disproportionately inequitable ways that we admit students to college or allow them to go into careers. We’re working on those things.

(00:21:21):
And Greg, and I really want to put this on the table of, hey, assessments have to be there too. If we’re going to rebuild education in the way that not just it’s been done, but the way that we want to see it, that aspirational view in the future includes probably doing everything a little bit differently, like handing this off to the next generation better than we got it. And if that doesn’t include exams, then we’re missing a big piece.

(00:21:45):
So I hope that when people read this, and hopefully when they read the research work we’ve put into it already that they’re saying, “Oh, okay, I see how if not this style would work for my class, I see how I could seal this piece or this piece to make it my own and to come up with something that treats all the students in my class 2% better or reaches a 2% of my class that maybe I didn’t reach as well.” And so we hope that what’s coming through is a lot of aspirational, there’s more room to grow, there’s more places that we can take this ambitious science and health teaching, so that the future’s even better.

Kevin Patton (00:22:27):
We’ll be right back with more.

Murray Jensen, HAPS Hero

(00:22:32):
This spring at the annual conference of the Human Anatomy and Physiology Society, HAPS, my longtime friend, Murray Jensen was presented with the highest honor bestowed on faculty teaching human anatomy and physiology, the HAPS President’s Medal. It’s kind of like knighthood, meaning that it’s a big deal. Murray has been mentoring and otherwise helping A&P faculty in his home state of Minnesota as well as across North America and really around the world for decades.

(00:23:10):
Besides all his good work, Murray is a genuinely good person and always brings a bit of warmth and cheer wherever he pops up, which seems like everywhere, all the time. Every time I turn around, there’s Murray. He’s involved in this and he’s involved in that, and he’s just hanging around, chatting it up with everyone. Well, cheers and best wishes to Murray Jensen, this year’s HAPS President’s Medalist.

Our Motto: Be Prepared

(00:23:46):
We’re back with Ben Wiggins and Greg Crowther. In the last segment, Ben was talking about being a coach. You bring up this idea. I love the coaching analogy. I often use that with my own students in my own class as well when I’m trying to be transparent. And transparency is one of the things that you bring up in your essay. I don’t want to spill the beans or anything, but we hear about transparency in other contexts as well, but you’re applying it to exams.

(00:24:16):
One of the things that you’ve brought up already in this conversation, but I noticed also in the essay is that these exams, I think sometimes as instructors, we don’t always appreciate the life altering effects that exam scores have because exam scores are often a big part of the course grade. And those course grades, those are hoops that need to be jumped through in a way. If you can’t make it through that hoop or using the basketball analogy, you didn’t sink any baskets, sorry, you’re out.

(00:24:57):
And so here’s this person who maybe has spent the majority of their life just knowing that they wanted to be a nurse or a physician or a physical therapist or whatever it is, and they have to take anatomy and physiology, and it’s a challenging course for them, and they don’t get that B or better or whatever it is at your institution that you need to either get into or stay in the program or the major that really is the foundation of how you envision the rest of your life unfolding.

(00:25:30):
And what are you going to do? Are you going to try it again? Is it obvious that that’s never going to happen for you? What’s going to go on? And I discuss that with my students near the beginning of the course when the opportunity comes up, and it always does when we start our testing and preparing for an exam and so on.

(00:25:52):
And I tell them, “Okay, I know what you’re thinking, that if I don’t get a good grade on this exam, I won’t get a good grade in the course. And if I don’t get a good grade in the course, I won’t be able to go into this program or stay in this program that I want. And if I can’t do that, then maybe I’ll never have a career. And if I don’t have a career, maybe I’ll never get any kind of a job. If I don’t get a job, then I’ll be out on the street. And if I’m out on the street, I might get hit by a bus and die.”

(00:26:18):
And so they’re approaching this in a very highly stressful story that they’ve created for themselves that is kind of ridiculous. I mean, everybody’s snickering and what. And thinking, “But really, why are you stressing out so much if you’re not giving so much weight to this?” And I understand why there’s so much weight because we as an institution do that to our students. And so how can we remove some of that pressure?

Ben Wiggins (00:26:50):
Yeah. Oh, man.

Kevin Patton (00:26:52):
Go ahead.

Ben Wiggins (00:26:53):
Sorry, Kevin, I just want to add onto that that I see that telescoping going down to the word on exams. I’m talking about this with lots of professors where students will walk into the room and open up the page and not understand the first phrase of the first question.

(00:27:10):
And then you see it like, “Well, if I don’t understand this phrase, I’m not going to understand these three sentences. If I don’t understand question one, I’m going to get a zero on page one. If I don’t get any points on page one, this exam is an F for me,” and right into the full telescope.

(00:27:25):
And that’s happening in seconds with high stress situations, but surprising high stress situations. I just think you’re so right on in how you think about this. I can’t even describe how many times I’ve seen that exact thing happen to students.

Greg Crowther (00:27:40):
Yep. I wanted to add on too, because I think … I teach human anatomy and human physiology as separate courses in my case. And so for human physiology, it’s the second to last biology course before they go into nursing school. So it is a literal and figurative gate that they must cross or a hoop that they must jump through. I guess that would be more figurative.

(00:28:05):
So I’m very conscious of I am standing between this student and nursing school, so boy, my tests better be reasonable in allowing all those who can do it to pass through. And so I think sometimes we collectively badmouth our students by saying, “Oh, they just care about the grade.” Well, I think it’s totally reasonable if they need a certain grade to fulfill their career goal that well, of course they care about it.

(00:28:37):
So I think what Ben and I and others are trying to do is to make sure that the students’ self-interest of I need this grade corresponds to our interest of I want you to learn this material. And we can adjust our tests so that if they know and understand the stuff that we want them to know, they get the grade that they want, and then everybody’s happy, hopefully.

Ben Wiggins (00:29:02):
Bingo.

Kevin Patton (00:29:05):
We have this huge barrier, and we can do some things to make that easier. We can help our students reduce that stress by preparing them in the ways that you guys just mentioned, right?

Ben Wiggins (00:29:22):
Yeah. A student in Greg’s class, the night before the exam if they’re really stressed, they can go to test question templates and they can say, “Well, I’m going to see this kind of question likely or possibly. Here’s how I build my own practice version.”

(00:29:39):
And they can get a couple of students together and they can work with practice versions that they have generated. And if that stressful energy makes them go through 20 practice questions, they’re going to nail that type. And it allows the stress and anxiety to go somewhere productive for the parts of anxiety and stress that can go somewhere productive.

(00:30:02):
And I think both of our styles are really trying to bring that, let’s not choreograph, but let’s mold the pre-exam work in some productive ways so that it’s not just show up, and at least in some students’ minds it’s show up and either get lucky or not. And we’re saying, show up and let that work really come to the fore.

Greg Crowther (00:30:26):
I think this might not yet be totally clear to your listeners, Kevin, but the fundamental thing that Ben’s public exams and my TQTs have in common is that parts of the test are essentially pre-released in advance. So we have pre-released materials to acquaint students with this is the format of the questions, and these are the kinds of things that you’re going to have to do. And both of them keep some stuff hidden, but a lot of the test is revealed in advance so the students know what to study and how to prepare, and they can essentially start taking the test in a way.

(00:31:06):
So we feel like that’s useful on the stress front. They have a better sense of what the test is going to look like, so they’re less freaked out by it, but also it means we can ask them more interesting questions because we’re giving them this substrate to start working on in advance. So we can be very rigorous in our expectations, but also be fair because we’re giving them lots of time to prepare and start in on these complicated, interesting questions.

Ben Wiggins (00:31:40):
Totally.

Kevin Patton (00:31:43):
I see exactly where you’re coming from, and I really agree with that approach. And in my own courses, I’ve done what I can over the years, tried different ways of getting that to happen. But one thing that keeps popping into my head, because I’ve done all my learning in a particular background and style of teaching and learning, and I’ve taken courses in pedagogy and educational psychology and things like that in the way olden days when those were first invented, but I learned a lot about these things.

(00:32:22):
And so they’re rolling around in my head and they pop out at times like this when I’m thinking about, well, how can I prepare my students better for an exam? And so I think about, well, maybe I can give them practice questions, maybe I can do various things like that. But it keeps coming up and trying to stop me, I guess, or challenge me at least is, but that’s not fair. That’s cheating.

(00:32:48):
When you give students sample questions ahead of time, when you let them look at old tests or old exams, when you give them questions like the ones that’ll be on the exam so that they really have a great idea of what to expect in general, then isn’t that cheating? Isn’t that giving them a leg up that they shouldn’t have? So how do you guys react to that because I hear it from other people too, it’s not just me. But even when I say, “Well, yeah, I let them see questions like will be on the exam,” or, “I put my old exams out there for them,” or …

(00:33:28):
I even at one university, somebody, I can’t remember which department it was, but they contacted me very worried because they said, “I guess you don’t know this, but all of your students have all of your old exams.” And I’m like, “Yeah, I gave them to them. Here, you want the link? You can go to it and get them.” And they’re like, “What? We don’t do that.” I’m like, “Well, I do.” So you get some pushback on that because of that mindset. So how would you respond to that kind of a mindset?

Ben Wiggins (00:34:11):
I apologize if this is heavy-handed, but is it okay if I do a quick example here?

Kevin Patton (00:34:15):
Sure.

Ben Wiggins (00:34:16):
Let’s say you wanted to give somebody a test about this podcast and you say, “Okay, I want them to know about the topics we’ve talked about. I want them to know some of the major concepts.” Okay, so here’s a question we’re going to give. On the exam, I’m going to write what was the topic of podcast number 112? Well, that’s a question. That’s legitimate. That’s actual information. And you could give that to them as a surprise, and some students will know it and some students won’t, and we worry that who does or does not will be really filled in by some parts of dis-equity.

(00:34:53):
So one thing you could do is you could scaffold that question. You could say, “On the exam, I’m going to ask you about what is the topic of podcast number blank.” And then what your students will do invariably is they will go memorize the topics of every one of your podcasts because they know they’ll get points for it. And you and I probably both feel a little like, “Ah, it feels like I just gave it away.” They had to put in some effort, but how much are they really taking forward? And I agree on low level questions like that, it’s not fulfilling.

(00:35:30):
But what if the question was, we really want you to understand the A&P podcast philosophical framework around say, assessment, and we wanted to know what you really think. So the question we could give to students is in a paragraph or less, write what you think The A&P Professor would say in answer to this question, and the question is going to be withheld. So we could tell students it’s going to be some question about a major repeated topic that’s come up in many episodes of this podcast.

(00:36:06):
Well, that gives students a ton of meta information. They know how many points it’s worth. They know what they’re going to do. They know that they can practice their writing. They know how to generate examples and then try those out. And sometimes they’re going to show up to your office hour saying, “Hey, I tried this version of your question and I came up with this answer. How did I do?” They can go to tutors and ask that. They can really work deeply into a question that’s meaningful.

(00:36:34):
And because it’s not hitting them in some surprise mode, they can go right into writing this thing and then you can see them as students trying out your voice on this important deep, complex, conceptual question. That’s the kind of thing that TQTs and public exams try to get students to do is let’s scaffold higher level, more interesting cognitive questions to show us what students really know. And we’re trying to remove the logistical barriers of how much can you actually show up and be able to read cogently in a timed exam.

(00:37:12):
It would be a very, very complex question if you couldn’t ask questions about the question beforehand, but in this pre-release style, students get it. They say, “Oh, okay, I’m going to be writing like this person. I need to pay attention. I need to take notes. I need to learn more about this podcast, and then I can give you my impression of what this podcaster would think.”

(00:37:34):
I know that’s just an off-the-cuff example, but I think that helps. Well, hopefully it helps explain a little bit of why we’re not worried so much about the memorization of the lower level terms and why you can still assess people on what they know and what they can do.

Kevin Patton (00:37:52):
Yeah, I get that. That’s a really good example, I think. And I’m glad you wove in there, both the lower level questions and the more critical thinking style of questioning and really getting to what does the student know and what connections can they make so that they can think about it ahead of time. Because just being faced with that right on exam day, right at that moment when you’re under all this stress, you may not be able to think very clearly. But if you’ve thought about it before, then you’re thinking, “Okay, I’ve thought about this kind of thing before, so how would I apply that here in this particular situation?”

(00:38:31):
But something else that I think is really important to remember too is in anatomy and physiology, there’s a combination of both kinds of knowledge that at least I expect of my students. So for example, I give them a list of bones and bone markings that they need to be able to identify on their practical.

Ben Wiggins (00:38:49):
Same.

Kevin Patton (00:38:50):
So a lot of that is just memorizing where it is, going and looking for it. I think it’s a little bit beyond simple rote memorization, but not much beyond that. But knowing that ahead of time, okay, here’s a list of questions, where is, I don’t know, the manubrium, where is the femur? Where’s the head of the femur? So you can have all the questions because I basically give that to them ahead of time anyway.

Ben Wiggins (00:39:18):
Exactly. Exactly.

Kevin Patton (00:39:19):
Here’s your list of things that you could be able to identify. So we’re already doing that, and that’s a standard practice. That’s not anything newfangled at all. It’s just those higher level thinking ones, but the way you explain it of, well, we’re not giving them the question, we’re giving them samples of that question so that they can think through how to solve that before they get there.

(00:39:44):
I mean, it’s like in a math class or sometimes even in physiology, if you’re having your students solve mathematical problems, if they haven’t practiced working them out, then putting that on the exam, that’s going to be a disaster. Because even if it’s a simple calculation, if you’ve never done it before, it might really throw you off like, “Whoa, I never saw that before. What am I going to do?”

Greg Crowther (00:40:11):
If I could reach for a possibly useful analogy, one of the things I’ve heard from other people is that the ideal professor or the ideal instructor is a quote, unquote, “warm demander.” In other words, they are very demanding that they have very high expectations for their students, but they’re also very warm in supporting the student and being there as a resource for the student and helping the student.

(00:40:38):
So I think that’s what we’re aiming for, is to make those standards appropriately high so that straight memorization is not all the students have to do, but with these practice examples and templates to give the students the support they need to meet those high standards. And warm demander in turn reminds me of the coaching idea. So I think these are all related concepts in my mind at least.

Ben Wiggins (00:41:05):
Well said.

Kevin Patton (00:41:06):
Oh, go ahead. Sorry, Ben.

Ben Wiggins (00:41:08):
No, well said. That was brilliant.

Kevin Patton (00:41:11):
Yeah, I love that too. Hey, we’ll be back in a jiffy.

The Corner Pub

(00:41:18):
A few years ago, I asked a couple of podcasting experts to take a listen to an episode of The A&P Professor podcast and pick it apart. They did a good job of it and gave me some actionable advice that I think really did improve your listening experience.

(00:41:38):
One of the things that they particularly liked about this podcast is that it reminded them of one of those bars or taverns or pubs where nearly everybody from, oh, I don’t know, the firehouse or the courthouse or the newspaper across the street comes after work to talk shop over a mug of root beer or a tankard of ginger ale. That is, it seemed to them that this would be a great place for A&P professors to hang out at the end of the day and listen to what’s going on with their friends, to meet new friends, and to have a chance to ask questions, or to blow off steam.

(00:42:23):
Why not share that experience with a friend? Bring them along to The A&P Professor by simply telling them what you like and asking them to search for The A&P Professor in their device’s app store or wherever they listen to audio. It’s that easy.

The Next Big Leap: What Is It?

(00:42:45):
We’ve been chatting with Greg Crowther and Ben Wiggins about A&P exams, and in the previous segment, Greg had mentioned the idea of striving to be a warm demander. Actually, that term warm demander has come up on a podcast episode before. Do either of you know which podcast it was? Was it 112? 118?

(00:43:12):
I love the framing of the question that way because I don’t know. Ben, you mentioned episode 112. I don’t know what was in 112. I mean, when I see the title of it, I’ll think, “Oh yeah, now I remember what we talked about.” But that’s pretty useless information really.

Greg Crowther (00:43:26):
Could I make a point here?

Kevin Patton (00:43:27):
And sometimes, don’t we do that on exams? I mean, I’ve done that before.

Ben Wiggins (00:43:31):
Yes. Yes.

Kevin Patton (00:43:32):
I’m asking this question, why? What use is that? How is that student … Why do they need to know that? They don’t need to know that.

Greg Crowther (00:43:41):
I’m just going to say it was Krista Rompolski discussing a recent paper on the use of eponyms in anatomy. That’s my BS-ing answer that I write down when forced to say something.

Kevin Patton (00:43:54):
Well, you can tell that you’ve used that template approach, Greg, in your thinking, because that really does both that topic and the fact that it was Krista Rompolski discussing a paper, those are both common things that happen in our lineup of episodes, so very good. I’ll give you partial credit for that.

Greg Crowther (00:44:18):
All right, I’ll take it.

Kevin Patton (00:44:20):
Because you need the points, right?

Greg Crowther (00:44:21):
Yep, I do.

Kevin Patton (00:44:22):
It’s all about the points.

Greg Crowther (00:44:23):
Yeah.

Kevin Patton (00:44:24):
Okay, so another thing that occurs to me as I think about exams and was reading your paper and thinking about what you guys have already written that I’ve seen, I think what you’re advocating is a much more conscious approach to exams. I think it’s probably stating the obvious.

(00:44:43):
But the thing is, is that so many people, including myself at times, we do exams as an afterthought. Here they’re incredibly important for the student. They’re incredibly important to give us some feedback on how we’re doing in terms of how we’ve designed the course and how well we’ve helped our students learn things. And then we see the results of the exam and how many of us don’t feel horrible when everybody does badly on the exam, like I failed them, but of course they feel like failures as well.

(00:45:18):
And yet I look back and think, “Well, you know what? I should have put more effort into making these questions more meaningful or more tied into what I wanted them to know, what the learning outcomes that I had established for them.” But so many of us, even if we’ve learned otherwise, even if we’ve learned that it all has to be well-designed and connected to our learning outcomes and our overall objectives for the course, in real life we often don’t do that.

(00:45:49):
We spend so much of our time preparing our lessons, whether that be a lecture or a demonstration or an active learning activity or probably a combination of all of those things. We spend all of our time on that. We spend a lot of time executing that, implementing those lessons. We spend a lot of time helping our students understand those lessons when they ask us for help. And yet when it comes to the exam, we’re just like, “Let’s get this done. Let’s get this ready. I got to have my exams ready.”

(00:46:27):
Are there any tricks or mindsets or mantras that you guys have for making sure we don’t do that? Or is it just like, let’s just tape it on our mirrors, so every morning when we get up and brush our teeth, we see the little thing like, think about the exam, think about the exam, or work on the exam, work on the exam?

Ben Wiggins (00:46:50):
Yeah, I mean, it’s such a good question. Sorry, Greg, I hope it’s okay if I jump in here. It would be great to just, quote, “backward design” and have it just work, but it doesn’t just work like that for real people in real jobs. Professors are busy. And every “I’m busy,” it doesn’t mean I’m busy because I filled out some paperwork. It’s like, “I’m busy because I had time to give to a student.” We are crucially busy.

(00:47:19):
And so the tricks and techniques and tools to make exams better and fit within a real life, that’s what we need. We do not need, in my humble opinion, more theorizing about what somebody with unlimited time and unlimited help and effort would do. It shouldn’t even be part of the conversation because it’s just a distraction.

(00:47:41):
I really do think that the next great step in what exams should be, the next big leap is probably a method that someone out in the community is using right now and the rest of us just don’t know it. And that’s a big part of why we want to get out there and start figuring out what do people actually do in exams, not just anecdotally, but getting our survey out there, trying to find what people’s real work lives are like.

(00:48:11):
How do you do these logistical steps? How do you connect with students on a day-to-day basis? So that we can take the best stuff and distill it out and make that more available for everyone else. There’s just so many people doing exams in so many different ways, and we have both been in weird enough positions to see many of them, but still a tiny minority of what’s out there that we really want to get out there and see what people are doing.

(00:48:38):
So we can find those techniques, figure out what about them works the best, and get those out into more classrooms and make life easier for more hardworking professors that are trying to do these things and just might not have the step C to D little piece that they need to make that happen.

Greg Crowther (00:48:57):
I was going to try and add on something, which I don’t know if this qualifies as probably not a mantra but too many words. But I think if we accept the idea that we want the exams to be more connected to what the students are doing in class day to day, I think you can make a bit of a game out of it in a way that’s fun for you and fun for the students where you first as the instructor envision, okay, what do I want the exam to look like?

(00:49:32):
And then thinking about your daily lessons and interactions with the students, just start thinking about what can I foreshadow about these exam questions and let me have some fun revealing that to the students and talk about how could I ask this on the exam? What are the other variations of this kind of question? And the students will be engaged because it relates to the actual exam they will take later, and you feel better because you’re being transparent and you’re not doing this weird hide away what the real exam is going to be about.

(00:50:06):
So I feel like it does take work that’s unavoidable, but maybe it’s motivating to help you do the work if you feel like I’m creating a better classroom experience for myself and the students because we are really more directly and more explicitly getting ready for the actual exam down the road.

Ben Wiggins (00:50:30):
Totally. When I consult with professors or departments, it’s virtually always, hey, you are doing this thing that you really like. Let’s do more of that, not let’s change everything.

Kevin Patton (00:50:43):
I like that foreshadowing thing too. That keeps it on our minds as well, the exam questions. Because I know one way that I would foreshadow with my students is I used clicker questions. And not all, but some of my clicker questions would be, okay, we’ve just discussed this concept. I’ve just worked through some what if scenarios for you based on those concepts. Now look, it’s time for a clicker question.

(00:51:12):
And I give them a question that is a version of what they’ll see on the exam. I have occasionally used a question that I know that I’m going to use that question in its present form on the exam and I give it to them. And then the beautiful thing about clickers is we can see how everybody did and we see that not everybody came up with the same answer.

(00:51:34):
So I intentionally pick items sometimes that are the more challenging ones I’ve used on an exam before, so I know that students are going to have difficulty with it. But after analyzing it and seeing, no, that’s a good question, it’s not badly written, it’s not unclear, it’s not based on anything they haven’t learned or shouldn’t be expected to know. So that’s a good question. It’s just a hard question.

(00:52:01):
So I put it up there and then almost everybody gets it wrong, and you can see the graph. And then I’ll say, “Well, somebody who answered this wrong answer over here,” I don’t say the wrong answer, “but this answer here, why do you think you got it that way?” And then somebody else will pipe up and like, “No, that can’t be right. It’s got to be this. No, no.” In an ideal situation, they’ll all start yelling at each other, which that’s great when that happens. It doesn’t always happen.

(00:52:35):
I always leave the polling open when that happens. So you can see people start to change their answer to the right one as we work through it, and then pretty soon everybody’s on the right answer and they know that I do that because I tell them ahead of time, “Some of these questions, you’re going to get something just like it on the test. So these aren’t just to waste time here or to take attendance or whatever. They’re the kinds of things that you’ll be challenged with later on so that …”

(00:53:03):
There are many ways to foreshadow. And circling back to what you were just saying, Ben, about the reasons why you’re doing this survey. First of all, I see what you did there. It took us back to one of the main goals of this episode is to get people to participate in this conversation by doing that survey. And one of your goals in this survey is to hear all those little things.

(00:53:30):
And in doing this podcast over the years and many other things I do in the world of anatomy and physiology teaching, I hear a lot from people about things that they do over a cup of coffee or a beer or whatever, just chatting for a few minutes as we pass by each other. They’ll mention something like, “Oh my gosh, I never thought of doing that.” And often they’ll look at me like how could you have never heard of that before?

(00:53:57):
Well, it happens and I think a lot of us hear things and it just doesn’t register until that one perfect moment when everything’s in the right place for us to actually get it. And there are so many people out there with those little gems of things that they do that maybe other people haven’t heard of. And somebody can take that and either use it as is or integrate it somehow into what they’re already doing and maybe change or tweak something that they’re already doing to make things more effective or more inclusive or better in some way for their students.

(00:54:35):
So I’m really anxious. At the end, you can optionally sign up for what the results are to get news on what you found out. And so I signed up for that and I’m anxious to see that. I’m encouraging everybody to do that for my own selfish reasons. I want to know what everybody is thinking about exams. I want to know what everybody has tried with their exams because that’s going to give me some good ideas.

Greg Crowther (00:55:05):
Thanks, Kevin. So that URL, I’m sure you’ll put it into the show notes, but we can just orally read it off as tinyurl.com/stemexamsurvey should work for you. And I tried it with and without capital letters and doesn’t seem like the capitalization matters. So tinyurl.com/stemexamsurvey. We are actually focused on A&P at the moment, but that URL reflects the fact that we have some broader ambitions about maybe eventually surveying non A&P people.

(00:55:40):
I want to also add that in this episode, you’ve been hearing me and Ben voice lots of opinions about how we think exams quote, unquote, “should be done,” but here as we return to the survey, I want to emphasize that we’re really trying to just see what people are up to and not be overly judgmental about that.

(00:56:03):
And one of the ways that we’re demonstrating that is we’ve committed, once the survey is done to submit this paper to HAPS Educator the official journal of HAPS. So in other words, we’re not going to take the HAPS member’s data and go tell on them in some other forum and say, “Oh, look what these horrible HAPS people are doing with their exams.” No, we’re soliciting your input and we will report back to you, the members of HAP and friends, what you said about exams. So we’re trying to keep it really rooted in the community-

Ben Wiggins (00:56:40):
Absolutely.

Greg Crowther (00:56:40):
… and again, as judgment free as one can be, hopefully.

Ben Wiggins (00:56:49):
We’ve got a pretty good data set already going, and I feel like we’re going to be able to pull out things that we can walk into a department administrator’s office or a dean’s office and say, “Look, if you could give your faculty more time, they would be trying to do this and this and this. But in busy lives where we’re all teaching five sections, those are hard to do.” How do we make space or how do we really honor that equity-based innovations that people are going to try? I mean, it’s self-serving for me to say that, but I think at its heart, it’s also student-serving and faculty-serving and that’s where we hope we’re going with it.

Kevin Patton (00:57:28):
Well, that sounds great to me because that truly is a challenge, the time situation as we said a few minutes ago. There are a lot of things I would like to do differently in my class, including how I do exams because I use mostly the automatically graded multiple choice and things like that. There are some ways you can make those things a little richer than just your standard multiple choice question, but I’d love to have more in the way of short answer and essay and so on.

(00:58:00):
But often, in the sciences especially, we end up with huge enrollments in one class session. And both of you teach at a community college, so we don’t have teaching assistants typically or anyone else to help us with that sort of thing. And so, man, time is a huge issue when it comes to those things.

(00:58:24):
So if we can have some data that we can show to our administrators and say, “Look here some things that other people are doing. Here’s some things that seem to be working very well. I’d like to try it, but I can’t under these circumstances. How can we sit down and together find a resolution?”

(00:58:43):
And maybe there are other resolutions than low class size, I don’t know. So if anybody knows a good solution to that, put that in one of those last questions in the survey and then we’ll all know. There’s so many people that have these nifty things and think it’s not so nifty, put it down there and it might be the next big thing that’s going to help a lot of us.

(00:59:07):
So yes, Greg, you’re right, I’m going to have that URL that you just gave in the show notes at the episode page. And thank you for giving it verbally as well because not everybody can check the show notes. And I really encourage you to, well, right now while you’re thinking about it or today, go ahead and take that survey. And like I said, it’s a very brief survey and it does give you the opportunity to spill your guts about exams too.

(00:59:38):
Greg and Ben, I would like to keep this going for a long time, but we have exams to grade and things to do. And-

Greg Crowther (00:59:46):
Well, come on, man, you’re going to give Justin Shaffer five hours on clickers. So I expect the same for my favorite topic.

Kevin Patton (00:59:55):
It was only four hours I was giving him, so don’t exaggerate.

Greg Crowther (01:00:00):
All right, all right.

Kevin Patton (01:00:00):
Well, you guys come back and we can talk about exams all you want and we can do a five or six, let’s do six. Let’s wrack up six. Oh my gosh, I don’t know if I’d be able to last that long as interesting as that would be. Maybe we can break it up into shorter pieces. But chunking, that’s another topic we could talk about.

Greg Crowther (01:00:20):
Mm-hmm.

Kevin Patton (01:00:21):
Once again, I appreciate you both coming here on the podcast to talk about your life’s work and where it’s heading right now with these exams. And I hope you do come back and we can talk about a lot more, especially when you get some results from your survey. That would be a blast to be able to discuss those.

Greg Crowther (01:00:41):
Yeah, happy to do that, Kevin. That’s a concrete and actionable idea.

Staying Connected

Kevin Patton (01:00:50):
Well, that was fun examining the A&P exam with Greg Crowther and Ben Wiggins, and they brought up all kinds of interesting ideas. I think a lot of them probably resonated with us. I know quite a few of them resonated with me. And a lot of them really got me thinking about ways that I’ve tested and ways I’ve heard about you testing in your classroom. But I don’t really have a complete picture of that. I don’t really know what everybody does in their classroom. And a good way to get at that is this survey that Greg and Ben are working on right now.

(01:01:28):
So I’m going to give that URL again, it’s a really easy one to remember, but just in case, go type it in right now into your device and take that survey. It really is a short survey. And when I said at the top of the episode that it’s fun, it is fun. In the way that this podcast got us thinking about these things, the survey stirs up some things too, that makes it a very interesting experience.

(01:01:55):
And I think that by the time you get to the end of the general questions of the survey, when you get to the open-ended part, even though you don’t have to fill that part in, you’re going to want to, I pretty much guarantee it. And they really do need everyone’s voice or close to everyone. Okay, they can’t get everyone’s voice, but let’s do what we can to help them out. Okay? I’m counting on you to help them out.

(01:02:21):
So let’s go to tinyurl.com/STEMexamsurvey. Once again, tinyurl.com/STEMexamsurvey. I’m just going to leave it there at that and not give you any more information so you can just quickly go to your browser and click it in and get her done.

Aileen Park (01:02:47):
The A&P Professor is hosted by Dr. Kevin Patton, an award-winning professor and textbook author in human anatomy and physiology.

Kevin Patton (01:03:08):
There may be additional side effects not mentioned in this episode.

(01:03:24):
Hey, I’m not kidding about this survey. It’s going to help all of us if we’re all good citizens of the A&P teaching community and do the survey. Once again, it’s tinyurl.com/STEMexamsurvey. I’ll see you down the road.