Episode 11 | SCRIPT

TAPP Radio Ep. 11 TRANSCRIPT

Test Debriefing Boosts Student Learning

Episode 11 | Transcript

Test Debriefing Boosts Student Learning

Introduction

Kevin Patton:                    We’ve all had that student that walks into our office and says, “I just can’t get a good grade on any of your tests, what do I do?” Well, you know what? I found a tool that may help.

Aileen:                                 Welcome to the A&P Professor, a few minutes to focus on teaching human anatomy and physiology with host Kevin Patton.

Kevin Patton:                    In this episode, I talk about how glial cells prune synapses in the brain, and then we talk about helping students debrief after taking a test.

Has a new organ been discovered?

Did you see or hear any of those headlines last week that a new organ was discovered in the human body? It’s called the interstitium, and it’s made up of little fluid spaces in the connective tissues in the wall of hollow organs like the submucosa and it’s found in the fascia of muscles and in the skin and in lots of other places in the body, and I don’t know if you’re thinking what I’m thinking about these, but don’t we already know that there’s an interstitium? Don’t we already know that interstitial fluid forms little pockets within some of the connective tissues of the submucosa and other areas of the body? Not only that, even if we had not known that before, would discovering them and identify them, would it constitute organ? Really? Isn’t it a part of an organ, really a structure within a tissue that makes up an organ?

I’m not the only one that has had that reaction, it turns out. A lot of scientists are pushing back and saying, “Wait a minute. This is not new information.” At least not in the way that it’s being portrayed in the headlines and even by the authors, that this is a new organ discovered in the human body. That’s taking it kind of way beyond what they’re really seeing here. We’ve seen those pools of fluid, of interstitial fluid for a long time. We know that interstitial fluid forms these little pockets. We know that eventually it drains into the lymphatic system, into the lymphatic capillaries and makes it way back to the bloodstream eventually.

You know what’s new here? Well, you know, they did some interesting visualizations. They used a kind of microscopy called endomicroscopy, where they take an endoscope and use that as a microscope and they can make images down to a depth of about 60 or 70 microns beyond the surface that the endoscope is at. So they did that and some other interesting imaging techniques, and so on, and pointed out that when we’re looking at histology slides in our course, for example, or in a textbook; the collagen bundles look denser than they would in life because these pools drain out and collapse.

Of course, we’ve known that for a long time. But it is, I think, important to go back and really think about all of those little fluid spaces being there, and the function that they might have. There is a growing awareness of all the different kinds of mechanical and chemical and even electrical interactions in extracellular matrix and tissues throughout the body. I think that this article contributes to that overall conversation that we’re having and increasing awareness of the dynamic nature of these tissues, which not all that long ago were thought of to be more or less glue or packing material or reinforcement material in the body, and not really an active dynamic area that is involved in so many different functions; it’s important for homeostatic balance in our body.

You know, I do have the link to this article. It’s an open article, open access article. I have a link to it in the show notes and on the episode page at the AP Professor.org. I have a diagram at the episode page, and I also have some other articles, including an article that rebuts the premise that this is a new discovery and that it’s even an organ. So have fun with it, and if you have any other ideas about it, give me a call on the podcast hotline. That’s 1-833-LIONDEN, L-I-O-N-D-E-N. Or send me an email at podcast@theapprofessor.org.

Featured topic: debriefing after a test

Kevin Patton:
If you’ve heard any of the previous episodes of this podcast, you know that I’m an enthusiastic fan of testing as a learning process. It’s not just a way to measure the progress of our students or how much they understand about the content of our course, it’s also the process by which they learn. And so, as I’ve described in pervious episodes, I give my students a lot of tests. They’re doing retrieval practice when they do those tests.

Today, I want to talk a little bit about a method that I’ve used for a very long time that has worked very well in enhancing that learning effect of tests. We can call it test debriefing, and it’s sort of a natural thing that I think all of us as students do after a test. We think about what went wrong on the test, if anything, and likely is going to be at least one or two little things, if not, a lot of things that went wrong on the test, and then we do some self-analysis and then we think about, “Well, what did go wrong? Why did it go wrong, and how can we prevent it from going wrong in the future?” So that’s what we mean by test debriefing.

I think a lot of us tell our students, “Well, go over that test.” Especially if we give them the test back, and I’m going to recommend some reasons why that’s a good idea in a minute. Even if we don’t give them back their test, we tell them to think about what went wrong on the test and prepare yourself … Use that information to prepare yourself for the next test. So yeah, that’s test debriefing, but what I’m suggesting is trying something a little bit more organized and possibly a little bit more helpful for our students.

So, I actually recommend a specific process. Here’s how I do mine, and maybe you can take something from this and develop your own process or copy mine, if you want. I give them a worksheet. It’s called the “Test Analysis Worksheet”, and there’s a link to it that’s there on your show note for this episode. So you can download this link or give the link to your students so that they can download it and use it themselves. You can either assign this to your students or you can just have them use it as needed.

The test analysis worksheet is basically a blank grid, and along the top, the headers of the rows have just the item numbers. Then it’ll have one column so that you can write, the student can write down the concept or the topic being asked in a question that they got wrong on the test so that they know what the question was about and you’ll see in a moment why that’s important. And then I have various columns, like in the next column, I’ll have the entry “misread the question”. The next one, “misread the choices”, so that would apply if it were a multiple choice item or a matching item or a put-in-order item that one or more of the choices were misread and that’s why they got it wrong.

So, they’re going to be checking off these columns. The other columns say, “They knew it, but they forgot it.” So there’s a retrieval issue there, is there if they felt like they retrieved that information, but weren’t able to on the test, whoops, let’s check that one off. The next column says: “It wasn’t in the notes.” That is in the study materials that student was using was overlooked for one reason or another.

Next column says, “Changed the answer”. There are some students that get it right, and then they go back and overthink things and they change it to the wrong answer. It doesn’t happen a lot, but when it does, you can check that one off. The next column says: “Thought it was unimportant”. So in other words, students sometimes make big mistakes in what they think is important or not important in terms of what they need to focus on in their studying, so that might be the reason they got that item wrong. They didn’t think it was important. Obviously, the instructor thought it was important enough to put it on the test.

Then the next one is, “Wrong order”. I use a lot of put-in-order items on my tests, in anatomy, I like to have students rearrange things in the correct anatomical order. It’s layers of the skin or the layers that constitute the wall of the heart or the order in which materials moves through the gastrointestinal tract or urinary tract. It’s a good way to get students to apply facts, and not just regurgitate facts or demonstrate familiarity. It demonstrates instead, mastery and mastery of the ability to apply that information. And sometimes things go wrong when you have them put things in the correct order. So when they get it wrong, they check off that little box in that column that says, “Wrong order”.

Then I have another column that says, “Too complicated.” In other words, maybe the test item was too complicated. That would be good feedback for me to have. I didn’t pose a very well constructed test item and it was too complicated, or more often that is referring to the fact that the information was too complicated and the student just gave up on it or they just really never did master it; which makes you wonder why they felt like they were ready for the test, but this is a good sign for them, right?

Then lastly, I have a little bit larger column that just says, “Other” because obviously, those things I just listed as things that could go wrong in that test item, are not the only things that could go wrong with test items. Those are just things that over my time in my courses, with my students that I found to be frequent reasons why a student would get an item wrong. So, what I have the students do is take a blank worksheet, a test analysis worksheet, and put their name at the top, which test it is that they’re analyzing. I give multiple attempts on my online practice tests, so I’ll have a little space for which attempt it was so that it can get filed along with their test correctly.

What they do, is they go through their test item by item. Now, most of my tests are online. Some are … The only tests that I give that are in class are my mid-term and final exam. The online tests they can print out, or just leave on the computer and analyze it there. I have my learning management system set so it will mark which items are wrong, but it will not give the correct answer. I want my students to find the correct answer. I feel like that’s part of their learning process.

Now, they can always double check with me. My first question will be, “Well, what do you think the right answer is?” And I never, ever just tell them what the correct answer is. I always lead them through that process by guiding them, asking them questions, and so on. But they can check with me to ensure that the item is correct or not. On their mid-term and final exams, I give them back, and I strongly encourage all of us to do that, to give the test back so students can do this test analysis with all of the test editing software and so on that is available nowadays, there’s no reason why we can’t give our classes multiple versions of a test and then change those versions out from semester to semester or year to year.

Yeah, it takes a little bit of extra effort, but these days, with technology that’s available, it doesn’t take that much extra effort and it’s well worth it in terms of student learning to actually have something that they can analyze in their hands, that test and actually learn from it. Remember, my whole approach is that a major part of testing is the learning process. If they don’t have those test items available to them, then they’re not going to really be able to get the full benefit of that learning process.

So, once they have the test in their hand and they have this blank test analysis worksheet in their hand and they’ve marked which test it is; so when they go through item by item and find all the ones that are marked as incorrect. And then they will fill out that row in the table. So the first one, let’s say it’s a test on the bone module, the skeletal system. Item number one, they write down for the concept, ossification. That was a question about ossification. So they might check off that they misread the question, they might also check off another column if something else went wrong. For example, wrong order. So maybe the misread the question, they put things in the wrong order, and so both problems existed for that one.

Then they go to the next one they got wrong, and maybe that was a question about generalized structure of a bone, let’s say a long bone. So they write that down under the concept bone structure and then maybe for that one, they misread one of the choices that was given to them. And then they’ll go down and maybe they’ll have some other ossification questions they got wrong or other bone structure questions. Or maybe it was on bone development or maybe it was on calcium storage or calcium homeostasis.

They’ll go through and they’ll make all these check marks, and then when they’re done, they can look for patterns. So the process of going through it in the first place is very useful, because it forces students, in a systematic way, to identify exactly what went wrong in each question, and that process, in and of itself, is going to help students figure out what went wrong and then that leads them to why it went wrong. Then they can start to develop a mindset for trying to figure out, “Well, how can I fix this problem?” But we’ll get to that, come back to that in just a second.

Let’s move on here. Looking for patterns. So in my hypothetical example, maybe they missed three questions about ossification in their test. If they circle those and say, “Wait a minute, there’s a pattern going on here, ossification is a topic that I really messed up on.” So now, they have even more information. They can see that there are particular topics that they were weak on, and that can lead them to go back in how they prepared for the test. Maybe it was a problem in their studying process. Maybe it was something that they missed along the way, they just didn’t participate in that part of the course for whatever reason. Maybe they were sick and didn’t realize that they skipped a part or who knows what? But something was missing there.

Or maybe they realize like, “Yeah, I never really felt confident about ossification, and yet I never got any help with it. I never sought help from my fellow students, from maybe experts at the learning center, I never went and talked to my professor about this to get extra help and further explanations about ossification, so my bad. I should have done that.” So that is helpful to know for their learning process, so they know that they made this kind of error, and hopefully that will prevent them from making that kind of error in the future.

And then if they total up the check marks or the X’s, or whatever they made in the various columns, they might see they have one in the column for misread questions and one in most of the other columns. But let’s say misread the choices, they have three or four items that are checked there. Now there’s another pattern that they can see. “Here’s a kind of mistake that I make a lot. So what can I do to prevent that?” Well, if it’s misreading the choices, there’s some pretty simple solutions that may work. One is: Keep in the back of your head that you should always go over the choices completely and carefully, and if time allows, go through them again. Maybe more than once or twice to make sure that they have really read and understood each of the choices that’s given there.

A lot of my students, on a multiple choice test, will find the first answer that seems good to them, and not even read the other choices, and therefore, they’re going to miss a section or miss a choice that was a much better choice than that first one they picked. My tests are constructed so that most of the multiple choice items, you pick the best one. There might be several plausible choices, but you need to pick the best one. So they can go through those various columns and add them up and see, well, what are the patterns and how to fix them.

That test analysis process is one that gets them to use a piece of paper, write it down, and go through things in a very organized, systematic way. It’s sort of like training wheels. If they do that again, again, and again, not only is it going to reveal to them some problems that a brief thought about how they did on the test would not have revealed, it’s also going to get them in the habit of doing this so that they don’t necessarily always have to write it down. They can just automatically go through and think about, “Okay, as I’m going through these wrong answers, what kind of pattern do I see appearing here that I can fix?”

This is also very useful as a group process. You can lead the students through this as the entire class group to think about what went wrong and what kinds of things could go wrong on a test, or you could have them get into small groups and go through their tests as a small part of your learning process in class, or maybe you assign them to do that in groups outside of class, or even just recommend a student and say, “Hey, here’s the test analysis worksheet, here’s the process. Go do it, and if you want to do it on your own, fine, but it may work better if you do it in your study group.”

Then students can kind of learn from each other, the kinds of things that can go wrong, and what each of them has found to be a good solution to fix the problems that are going wrong. Another area that this works very, very well in, is in that interaction between the professor and the student, whee a student will come to the professor and say, “I’m not doing very well on these tests. What’s wrong with me?” They expect us to diagnose them, and I want to be able to do that, but I think a lot of us aren’t trained in that. I know I didn’t get much training in that, even though I did have some educational training before I became a professor.

But I found that this test debriefing process is a very good tool for that. So when students come to me with that kind of a question, “What’s going wrong? What’s wrong with me? Why am I not getting the “A” I think I should be getting and only getting a “C” or a “D”? What’s wrong with me?” I tell them to show up with their last test, or maybe several tests, but also a filled out test analysis worksheet.

If they don’t have that with them, then I reschedule the appointment and say, “Well, I really can’t help you until you do that.” It’s sort of like visiting your physician and your physician had told you to go to the lab and get some blood work done. You show up and say, “Well, I didn’t have the blood work done, but can you tell me what’s wrong with me?” No, not really, not until I have the blood work that I need to make some good recommendations and a good diagnosis about what might be wrong with you.

So, that’s the lab test they need to do, and they bring that with them and then we can go through it together. We can lead them down that road of, “Well, do you see here that you got three questions on ossification wrong? Maybe that was an issue. So tell me about it, do you remember learning that material? Do you remember studying that material? What did you do to study it? How did you practice that? What steps did you take, and maybe there’s some other steps you could take, or different steps that you could take.”

Now you’re getting from the diagnosis part of it to the treatment part of it. What went wrong, why did it go wrong, how did it go wrong? And now, how can we fix that? How can we prevent that from happening again? Then we go through the other patterns. “Well, you’re misreading choices. What do you think are some ways that you can avoid that?” Every student’s mind is a little different, works a little bit differently. Sometimes, we run into students that are way off to one end or the other of the spectrum of how human minds work. And so, it takes a little effort to listen very closely to them to see what exactly is going wrong, and then think of strategies. Very often, students, if they’ve gotten as far as your A&P class, they’ve probably discovered some of these things about themselves already, and they’ve already learned some strategies, they just don’t realize they need to be applying it in this particular case.

So test debriefing, I’m suggesting, can be and maybe should be a little bit more formal, systematic process than just, “Hey, go over your test and see what went wrong, and then move on from there.” Maybe it should be a little bit more specific, and if nothing else, we can use this as a tool for those learning sessions when students come to us asking for help, and they don’t know what kind of help they need and we don’t know what kind of help they need. This might show us what kind of help they need and how to help them.

I hope it helps you.

Aileen:                                 The A&P professor is hosted by Kevin Patton, professor, blogger, and textbook author in Human Anatomy and Physiology.

Kevin Patton:                    Nothing I say on this podcast should be taken as medical advice or as solicitation to buy or sell anything, or as a recommendation for wine pairings.

Last updated: June 22, 2021 at 16:49 pm