In this episode of Status Check with Spivey, Anna Hicks-Jaco has a conversation with PowerScore Founder & CEO Dave Killoran about the removal of the Logic Games section of the LSAT, the new writing section, and what all this might mean for the future of the law school admissions.
You can listen and subscribe to Status Check with Spivey on Apple Podcasts, Spotify, YouTube, and Google Podcasts.
Full Transcript:
Anna: Welcome to Status Check with Spivey, where we talk about life, law school, law school admissions, a little bit of everything. I am Anna Hicks-Jaco, Spivey Consulting's President, and today I will be talking with Dave Killoran: an LSAT expert, the founder of PowerScore and author of the LSAT Bibles, one of my co-authors on the PowerScore/Spivey Consulting Law School Admissions Bible, and technically, my former boss, because I tutored for PowerScore a little bit while I was in college (though I never met Dave during that time).
Dave and I are here to talk about one of the biggest, if not the biggest, changes to the LSAT in decades, which is that this fall, August 2024, they will be removing the Logic Games section of the test. I'll let Dave explain more about what's going on, and then we'll talk about what you should do if you're currently considering taking the June LSAT with Logic Games or waiting until after the section is gone. How will the removal of Logic Games is likely to impact the overall score distribution curve—will scores go up? Will scores go down? How might that vary across the board? We'll also talk about the new LSAT Writing section also debuting this August and what this all means for applicants. Hi, Dave!
Dave: Hey, Anna, how are you doing today?
Anna: I am great! How are you doing?
Dave: It’s a little rainy, but no complaints in general.
Anna: So, we're here to talk about Logic Games. We're here to talk about the LSAT. Obviously there has been so much changing with the LSAT, especially around the pandemic and the LSAT-Flex and all of that. There was/is, to some degree, the infamous “bubble,” which we both know Mike hates that terminology, but—there being more high LSAT scores than there were prior to the LSAT-Flex, to some extent that still exists. But there's going to be more changes this summer and potentially the biggest changes—I’m kind of curious about your thoughts on that. So first of all, can you get our listeners up to speed, Dave, on what's going to be going on with the LSAT starting in August 2024?
Dave: Yeah, happy to do that. And hopefully, if they're thinking about applying to law school or taking the LSAT, they've heard a little bit about this. But after years of consistency with the composition of the tests and the content, they are taking away the infamous Logic Games section, also known as Analytical Reasoning. We knew this was coming. There'd been a lawsuit that kind of led up to this and they'd had a period of about four or so years to prepare for it. And we thought what was going to happen was they were going to trade it out for some type of new section. And they chose not to do that.
So a little bit of a surprise there, but what they chose to do was even more interesting. And that was to take the single-scored Logical Reasoning section that we have currently and double that and not really do anything else.
So the test currently is one scored Logic Games, one scored Logical Reasoning, one scored Reading Comprehension. Once we hit August and thereafter, it's a permanent change. The scored portion of the test will be two LRs and one Reading Comprehension. So they're doubling up on Logical Reasoning. And then you'll have an experimental section. That won't be Logic Games anymore either, or at least potentially Logic Games. That'll be either Logical Reasoning or Reading Comprehension. So they've gone very verbal-heavy here and removed that kind of puzzle aspect that Logic Games represented. So, really a huge change. One of the biggest in 30-plus years here.
Anna: Yeah, it seems like a massive change. I was about to call it the biggest change that we have seen to the LSAT. I think the LSAT-Flex had such a significant impact that I hesitated to categorically state that. But looking at the long term, this has to be bigger, right, because this is a permanent change.
Dave: Yeah, they've changed the composition of the test before. Like, we've seen sections come in and out. But Logic Games has been a part of this exam for over 40 years, and it's been a stable test, and what you allude to I think is actually a really big change. The Flex LSAT and then the take-home version of it, the online version, I think that represents a huge kind of like format and structure change. This is the biggest content change in 40-plus years. And the LSAT has changed. They've tried all sorts of items over time, but when you have a 40-year period of consistency and then all of a sudden that just goes away, I think that's the bigger news overall. But certainly, the online testing had its own pluses and minuses, I’ll say that diplomatically.
Anna: Yeah, but both very significant changes in the history of the LSAT, I think we can at least agree to that. I am going to say something very nerdy here. I am going to miss Logic Games. I loved Logic Games. I thought they were so fun. Which I realize sounds ridiculous, but I feel like, certainly Reading Comprehension, but then Logical Reasoning also to some extent resembles other standardized tests that you take, versus Logic Games is just this whole different sort of skill set. Even very, very intelligent people come up to a Logic Game for the first time without preparing and have no idea how to solve it in the allotted time. Most people can work it out if they had unlimited time, but of course you don't. So I found the process of learning how to do a Logic Game and learning how to get good at Logic Games lots of fun. And as you know, Dave, I used the PowerScore Bibles for my LSAT prep. That was my only LSAT prep; I self-studied with the Bibles. Never would have guessed that I would later on be an author of a PowerScore Bible, the Admissions Bible. Never in a million years would I have guessed that. That is pretty wild, right?
Dave: Very much so. But I think you make a really good point. It's not nerdy to say that you'll miss them going away. I definitely will miss them. The first book I ever wrote was the Logic Games Bible. And so to see that go off into the sunset, rather unceremoniously, it disappears and it's like, “Bye!” There's no award ceremony, there's no medallion or anything like that. It kind of hurts my soul a little bit. And part of the reason is that the variety of the test is now narrowed. What was a test that was broader and looked at different skills that could appeal to certain people and certain groups, that's now less so than it was previously. And I kind of lament that as well. But I mean, look, from a teaching standpoint – because you taught the LSAT – it's fun to teach Logic Games. It's fun to see the lights go on when somebody figures out, “Oh, that connects to this,” and they're able to actually deconstruct it. So I'm right there with you and I know that a lot of my fellow LSAT instructors feel exactly the same way.
We're excited to see how the test will change and what kind of changes we'll see inside the LSAT. But at the same time, it's like, I'm going to miss an old friend. There's no new friend coming in. It's just the same group of friends with one less that's out there. So I’m not a huge fan of this change Anna you know, as you might be able to figure.
Anna: Yeah, I'm definitely getting that. So now we’re instead of a new friend, we're going to have double Logical Reasoning, which of course, as you clearly know, it used to be two Logical Reasoning sections, and then they cut it down to one. Which I don't think anybody was broken up about not having two Logical Reasoning sections, especially when your experimental can also be a Logical Reasoning. So then you end up with three Logical Reasoning sections, which is again going to be a possibility here.
So going back to the two Logical Reasoning sections, let me ask what I think in some ways might be the most important question that I'm going to ask you for a certain subset of listeners. Which is, if you are someone who right now is thinking about taking the LSAT, thinking about applying next cycle, and you're not sure if you should take the LSAT before they eliminate Logic Games or after they eliminate Logic Games, what's your advice to people who might be making that decision?
Dave: Yeah, and I think that's a great question, because there are a number of people I'm sure who are listening or who will come across this and think to themselves, “I don't know what to do, which one is better?” The first thing I would say is if you're not familiar with Logic Games is to sit down, go to Law Hub and try a few sections of Logic Games. Just see what you think. Go in with, you know, an open mind. Sure, they caused a fear factor for some people, but many people loved Logic Games and became dominantly good at them. Go in and try a few sections and see what you think. You don't have to spend hours and hours, but certainly, an hour and a half would be great and then see how you've performed.
I'm not so much concerned about your initial score on the Logic Games section, I'm more concerned about how you as a test taker feel about it. Were these the worst things you've ever done and you never want to spend another minute with them? All right, that's going to send us in one direction. Or are you thinking to yourself, “This wasn't so bad and I could get into studying this”? Because the outcome here is that June is the last LSAT that will have Logic Games on it. You can still register for the June LSAT right now. If you feel that you are either going to be good at Logic Games or you can consistently become good, and this is probably the one section of the test that is most improvable in terms of studying and preparing, then take June.
If you get into that kind of dance with games where you're like, “I am not loving this, and this just is going to be a long slog,” then look to take August or later. Truly, if you're on the fence and you don't have a comfort zone, maybe try both. Sign up for June and see how it goes. And if you aren't satisfied with that score or the games doesn't go well or what have you, then you can go into August and feel pretty comfortable with it.
You don't have to make that decision now, but if you're uncertain, I would say tend towards June because that's your last shot with this. And I've had students who start off and they're getting 5,6,7 right in this section. And by the end, they're getting them all right consistently because they see how it unlocks. It's the most binary type of section and once you see a pathway, there is very little nuance to. It is all very almost mathematical in that sense. Whereas we know with Logical Reasoning and Reading Comprehension, it's much more nuanced and interpretive and that's looser. And so sometimes it's harder to get perfect at those sections than it is with Logic Games. So what we've been telling people is try it out, see what you think, and then make that pathway decision right there.
And right now you're in a great spot because you have that choice. Once we hit June, everybody loses that option and Logic Games are gone I want to say for good, but I'm hoping it's not for good.
Anna: Do you think they might come back?
Dave: No, I really don't. Although I'll be honest with you, the way LSAC has talked about Logic Games contributing to the validity of the LSAT in the past, if you go back and read their own research reports, they made a very big deal about how important it was. I'm not in love with the idea of removing something that in their own words was a huge part of the viability and utility of this particular exam. So not much we can do about it, obviously, but I could see in the future, ten years from now where they might say, “hey, this is losing a little bit of its accuracy. Maybe we should flip it around and bring it back.” It doesn't help anybody right now though. So we'll just have to let that one sail off into the sunset as well.
Anna: I totally agree with you that it has that utility, not just from the literature and from the studies, but even as we were just loosely talking about earlier, that's the section that feels the most different from other sorts of standardized tests. And there is a lot of discussion of the SAT, the ACT, those predict law school performance also – not as well as the LSAT – but is there going to continue to be that delta there where the LSAT is significantly more predictable when it no longer has Logic Games, it no longer has this big differentiating factor.
Going back to the advice that you were giving, I think that's great advice, especially with regards to how you feel about the section versus how you score, because that is huge. I think most people don't do well with Logic Games the first time they look at them. But I always said when I was tutoring for PowerScore, this is by far the most learnable section. So even if you take it and you get the vast majority of them wrong. You very well could be getting 1, 2, 0 wrong consistently after just a month or two of studying, even depending on sort of your natural aptitude for it and how much you're able to study. So very, very learnable section, but it varies based on the person. It's not necessarily for everybody.
Dave: Yeah. And just to add to that, if you are a person who does go in there and takes a section and just gets crushed by it, don't feel bad. This is the section that people who are taking the LSAT use to scare other people about it and to prove to their friends and parents and significant others that it really is a tough test. Very ominous the surface, on the face of it, when you first encounter it, it's like, “What's going on?” And then you think, “I can solve these.” And then you realize that you've already used eight minutes of time and that was all the time you had allotted. Once you start seeing patterns and the connections, it really does become something that you can do very quickly. It's like driving to me. You're not very good the first time you drive a car, the thousandth time you go in to drive a car, you're so much better. You're not even thinking about the little things. And that's exactly how Logic Games are. So you're making me wistful about a section leaving that hasn't left yet.
Anna: Here we are feeling wistful about an LSAT section. It is what it is. There's a reason that people translated Analytical Reasoning into a new name that includes games. You know, there is a fun element to it. But moving on a little bit. We've talked about how it is for many people the most intimidating, the most difficult section in the beginning, but how it's also the most learnable section.
That makes this question a little bit more difficult to pin down, I think. Do you think that eliminating Logic Games is going to cause LSAT scores to increase, decrease? Is that going to be different across the spectrum of scores? I'm curious your thoughts on how this is going to affect how the LSAT looks and how LSAT scores look overall.
Dave: And I think that’s a great question and it's one that as soon as this was announced, I started contemplating it and I vary a little bit in terms of how I see this. And part of the reason is because there's so many factors in play. We know some of them will be positive. We know some of them will be negative. Sometimes trying to assess the overall net effect or the degree of that effect is a real challenge. Overall, I'd probably say that we're going to see a decrease in top level scores. That's kind of like my net idea there.
What happens inside of that though, is that the composition of how people score is going to be radically different. One of the things that you see with really high-level scores is that many of them are really good at Logic Games. All those people are going to be affected. So the first reaction that a lot of people have is that scores are going down for sure.
But on the flip side of that, there's a number of factors that play the other way. There are people out there who are fantastic at Logical Reasoning and Reading Comprehension, and they can't get into the 170s because Logic Games is holding them back. All of a sudden, the people who lost – by having Logic Games removed – now you've got a group of people who are benefited by that. And so you're going to see a rise in scores from that particular group that's out there.
You also have this kind of time factor that's in there. Logic Games is certainly something that takes a long time to prepare for, and now you're removing that, so that's less time to study there because you're no longer working with three sections. So let's say you've got somebody out there who is working a full-time job, or a single parent for example, who probably could score better if they had more time, but trying to get through all three sections and study each one of them adequately has been a real challenge. Well, they just lost one-third of the studying, maybe one-half of it even depending upon who they are.
So now they have more time to focus on just two very related types of skills that they have to display during this test. And so you might see scores rise with that particular group. So I see all these little things in there. And even when you look at the composition of LSAT scoring itself, and you look at say the majors that people have historically, and they're correlating LSAT scores, a lot of the STEM majors score pretty well on this test. You're a mathematics major. Many of you are going to be pretty solid. Now, there aren't thousands of math majors taking the LSAT, but when you start getting into engineers and statisticians and so forth, all those students, I think, are going to be somewhat – as a general not every single one – but as a general idea, they're going to be hurt by this. Whereas we've got a lot of people who do heavy reading, whether it's political science, or English, or History, or what have you, who now don't have to worry about the Logic Games, more mathematical skills, they're being benefited. And I think from a long-term standpoint, we are going to see the scores come down just a little bit at the top.
But then again, with Logic Games gone, a lot of mid-level scores will probably be helped by that. So you'll see a shift inside that. And you'll see the various groups who are benefited, and hurt by this, change radically. I'm concerned about STEM majors going forward from this because one of their advantages in this process was Logic Games, and now that's gone. So we'll have to see. I'm very interested to track that in the future and see what we see from a compositional standpoint in law school classes. And that all goes back to the LSAT and the choices they made, but they must feel comfortable with it. We'll have to see whether or not they're right about it.
Anna: Yeah that’s really going to be interesting to see how things turn out because there are so many factors going in so many different directions. The point about studying is such a good one. I think as people who pay attention to the LSAT prep and admission space, mostly talking to applicants who are putting a ton of time and effort into this, because that's why they're talking to us, it’s easy to lose sight of the fact that there is a whole huge group of applicants who do not have the time to dedicate hours and hours of studying the LSAT every single week. That's just not, you know, a luxury that they have. So I think that's definitely going to be positive for those sorts of people.
Definitely seems like a potential negative for STEM majors and that group of applicants as well. One thing that I think that you didn't talk about as much, I think we are talking more about the top end of the spectrum, which makes sense in a lot of ways, and when we're talking about law school admissions, because those higher scoring applicants tend to make up a larger percentage of the folks who actually enter law school. But one effect that I was thinking before our conversation and then even more so as you were talking about the various considerations in your mind, was, I would not be surprised at all if we saw very low scores going up significantly. Because especially for those people who are not able to take a lot of time to study, they might stay at that stage of Logic Games where it's still just confusing and hair on fire, and you have no idea what's going on. I'm describing my mindset the first time I saw a Logic Game. So we might see low scores going up significantly is, one component of things.
And then the tippy top scores, I certainly could see going down because most people I know who were scoring 170 plus were basically getting none wrong or one wrong, maybe two wrong on Logic Games. I think most people who are at that top end of the spectrum are doing really, really well at Logic Games because it is more perfectible, as you were saying, than Logical Reasoning or Reading Comprehension. So totally makes sense to me that you might see those top scores going down a little bit.
But then you're right. There might be some middle movement. There might be some folks who would be scoring in like the low to mid 160s who now might be scoring instead in the high 160s to low 170s. It's so hard to anticipate exactly how those things are going to go.
So with all of these potential changes, I know we're talking about so many hypotheticals and so many different ways that things might go. What are your thoughts on how all of this might affect next cycle? Folks who are applying right now, it's not going to affect you at all, because it's the August LSAT, by that point you're too late, even for waitlists. Certainly, there will be plenty of applicants who have Logic Games on their LSAT that they took and that they're submitting to law schools. But we're going to start to see applicants who have this new test next cycle. So how do you think that's going to affect law school admissions? How do you think that's going to affect the cycle as a whole?
Dave: That's actually a really interesting point that you just made about the Logic Games that some people have and others don't. Because I think that's the first thing that people have asked me is like, “will they look at this new test differently?” Yes, they will to some extent because they'll be aware of the change. However, law schools have proven over time to not really care. They take whatever LSAC says is the score and they use that score. I think they'll be a little bit wary in terms of, ‘hey, how are the percentiles going to change?”
So I love that point that you make that there's going to be students applying next year, some with Logic Game scores, some without Logic Game scores. And of course the law schools can't see your individual section scores, they just see your ultimate LSAT score, but they'll know there's a difference. I think it'll contribute because of that uncertainty right there and what's going to happen. The first thing you'll see is a little bit of a slowdown. One of the things that happened during the infamous score bubble that Mike doesn't like to talk about is the idea that, “hey, LSAC said, there's no bubble, it'll go away.” And it didn't. And it caused all sorts of problems for certain schools. And so this time, whatever it is that LSAC wants to say is going to happen, or it's going to be the same, which is really what they're saying. And they haven't really released data that proves that beyond a shadow of a doubt. I think schools will take a little bit more of a circumspect view on this to say, “let's wait a little bit. We don't have to jump on this. Sure, this person has a high score, but we don't know whether we're about to see a lot more high scores or far fewer high scores.” So I think that'll be a big slowdown that happens.
If we're also right about the fact that high scores trend down a little bit, we'll probably see a change in the number of people at the 170s who are applying and maybe an increase in the number of people in the 150s and the 160s. So I think we'll see some median shifts. I could see a lot of schools trying to really hold on to the medians that they have, although they've spiraled to unforeseen heights in some respects, but you may see a lot of schools on the 25th medians or if their medians are in the 150s or low 160s, you might see those rise a little bit as well. So that could make some schools a little bit harder to apply to, and that could hurt splitters as well. If you've got a lower LSAT and a really high GPA, they might say, “well, look, our 25th percentile LSAT median has risen this year. So you're not as competitive as you were last year.”
And this almost highlights the entire scope of the change, there are so many little things that benefit some people, hurt other people. It really depends upon the particular circumstance of each applicant as to whether or not this is a good thing or a bad thing. It is not wholesale good, it is not wholesale bad and it will slowly play out. But I know you watch the numbers very closely in terms of the applicants and their scores, and it has been I must say very interesting to me to watch the percentage of applicants rise and yet the numbers of people in the 170s fall this year. Not that I'm a conspiracy theorist, but it almost feels like an intentional softening of the ground for percentages to drop next year. Because you wouldn't want to have a big drop next year and say, “oh look, they're very different than the prior year.” What you want is the prior year to look really similar, maybe that's just a happy coincidence for LSAC.
But it's interesting for me to see that we've got more applicants and lower top scores, because I think that lower top scores is likely to be what we see next year. Which means that at the top schools, say the T14, you would think that that would improve things a little bit and make it easier for applicants. So, that's a pretty broad answer because it tries to address a lot of different things, but hopefully that hits some of the key points as to what we might see going forward.
Anna: Yeah, I think it kind of has to be a broad answer, I think one easy simplified answer would probably be the wrong answer. Unfortunately, as much as I wish everything could just be an easy, simple answer, it's very interesting to think about how these changes are going to affect not only applicants next cycle, but schools next cycle. Like you were talking about law schools, especially at the top with these super high LSAT medians. Are they going to be able to retain those high LSAT medians? One, are there going to be enough applicants even to sustain them? Two, are they going to want to put in all of these resources and all of this effort to sustain those medians?
We've been talking about on our podcast rankings changes and the rankings methodology changes, they significantly decreased the weight of the incoming LSAT median and undergraduate GPA median metrics. Such that schools could increase or decrease their LSAT medians - depending on where you are in the rankings, it varies. But schools could increase their LSAT median by 5/10 points and not even go up in the rankings as a result. It's just such an insignificant component of the rankings as of right now, which I keep hedging because these new ones are coming out. They might increase, we will see. But if that stays sustained, then that's just another factor that law schools no longer have to be so worried about the medians. Because schools have really been forced to play this numbers rankings game because if you aren't paying attention to the rankings, if you're not trying to go up in the rankings, just by virtue of how the different metrics work and how some of them increase over time, you're going to go down in the rankings if you're not paying attention to it.
And even if you say outwardly, “we don't care about rankings, they don't mean a whole lot.” That doesn't mean that you aren't going to get fewer applicants. That doesn't mean that your current students aren't going to be upset. That doesn't mean that your alumni who are giving donations aren't going to be upset. So law schools are forced to care about this stuff, even if they don't want to care about it. If LSAT scores, especially at the top, come down, law schools might have to accept the fact that they are going to be seeing decreased medians. I think probably at least some law schools. I’ll call it right now, I think if we talk two years from now, we're going to see schools with decreased medians from where they are right now. So feel free if you're listening to this podcast two years from now to call me out on it if I'm wrong.
Dave: Some schools will certainly, as you said, try to hold on to those numbers. I don't think anyone's going to increase their top-line median even more, They're not going to have enough people. And you're right, I have spoken to Mike about the rankings before. I am very anti-rankings, because I think it's the tail wagging the dog in terms of U.S. News arbitrarily changes the numbers and acts as if it means something, whereas it's just really a spreadsheet. You're just moving things around. But we'll find out what it is that they do.
And it's a really good point to note that the weighting, that LSAT scores and GPA have really gone down and it might sound strange since I do this for a living and I actually love the LSAT as a test. I think it's a lot of fun. I love to teach it. I think it's a good thing that it actually becomes less important. It is just a test. It is only a few hours of your life. It is not the be-all end-all. And it certainly is not a signature of who you are as a person or as an applicant. And I think it's had an outsized importance for far too long. So them doing this I think it's really a good thing.
I don't know that U.S. News is doing it for altruistic reasons. I am not a fan. But at the same time, sometimes bad organizations do a good thing, and I think that's kind of what we have here with them is that. It's nice to see it become a little less important. If that means that this test change has less of a shock to the system, at least from the legal academy, then that's a good thing because this is just another test change. And we haven't really talked about the fact that LSAC - just the pace of change there as enforced by their president, starting back in like 2017, 2018 has been enormous in a field where change is not necessarily welcome or liked.
We run into these situations where it's like, wait a second, a lot of things are happening. And at a certain point, you even lose track of how much has actually changed. So I love what's happening with the rankings, decreasing that, I hope it stays that way, whether or not it will, I don't trust them, so.
Anna: Yeah, I feel like I have been talking about the rankings a lot recently, well with having a TikTok, which, you know, I'm constantly trying to talk about things from different angles, which now that the rankings are about to be coming out, it's something that I've talked about a good amount recently. And something that I have found myself saying over and over again is, “I strongly don't think” or “I strongly do think XYZ”, but U.S. News does things I really don't expect all the time. So, I take that with a grain of salt. They change things in ways that are unexpected to everyone who's paying attention with relative frequency. So how much can we really predict? But no, I agree with you. I think that it's a positive thing that the weight and the importance of LSAT and undergraduate GPAs has decreased a little bit. I certainly think standardized tests are meaningful and are important in some ways. But something that I have talked to so many applicants about over the years working in law school admissions consulting, is that for so long it has been significantly more numbers-focused than undergraduate admissions, than other graduate school admissions, than any other admission scheme that I'm aware of in the United States. And a big part of why that was the case was rankings.
So I do think that this is a positive in terms of just freeing up law schools to make more decisions about what they want to emphasize. I suspect there are still going to be law schools that are going to very much emphasize the numbers and that is what they care about and that's genuinely what they think, “This is what we want from our incoming class. These are our goals.” But I think law schools should be able to make that decision and be able to emphasize other things if that's what they think is best for their law school. So I think that's a positive. I think the commensurate increase in the importance of employment outcomes is definitely a positive. Obviously, law school students care a lot about employment outcomes. Applicants care a lot about employment outcomes. Everybody involved cares about employment outcomes. This is a professional school. You want to become a lawyer, right? So you're right. I think it might soften the change that this logic games elimination is going to impose upon legal education as a whole to some degree. We'll see.
Dave: The funny thing is that when you look at it, some schools will still be number schools because that's what they know. That's what they're comfortable with. They'll all fall back on that and I'm sure some people are listening, thinking why will U.S. News change the rankings at all? That's their business model is to literally change things every year so that it's not predictable for the next year. Otherwise, no one would bother. So we know there's going to be some degree of change because there is every year. How it all plays out will be really interesting. I know both you and I will have our eyes on it and certainly be commentating on it all over the place.
It's fun. It's interesting. It's kind of like watching a spectator sport. Who's rising, who's falling, who's happy, who's unhappy? So it's the People Magazine of the law school field.
Anna: Speaking of big changes, another section of the LSAT is also changing, which is less talked about right now for I think reasons that make a lot of sense. Can you tell our listeners about that?
Dave: I would be happy to, although I'm not happy about this change either. The LSAT Writing, which for years has been very consistent, using a kind of like prompt where you talk about two options and compare them to each other, which one's better, is now changing to what's kind of more of an argumentative format, where they ask you kind of like an open-ended question, and then they give you several different viewpoints in the example they have right now. And then you have to weave your own argument, make your own position, and then address at least one, if not more, of the viewpoints that they threw at you.
And it's a little bit less of a strengthen/weaken argument type of format that they're going to and more of a reading and make your own argument, make your own position. But there'll be a lot more front work on this, there's a lot more reading to do. So they're actually giving applicants up to 15 minutes of pre-writing, pre-reading time where they can take notes on the interface, and then you still have 35 minutes to write the essay out. It's still unscored, it's still delivered remotely at home on your computer and so forth. There'll be a little bit of a change in how they look at it. I think there'll be some initial interest, but it's not as if it's going to change the landscape.
What may change the landscape with that though is that LSAC said quite bluntly, “We're looking at scoring this in the future. We're going to use the data from 2024 and 2025 when we first administer this,” it starts with the August LSAT as well. “We're going to use that information and then we're going to think about scoring it in the future.”
John and I, we just did a podcast on this and we talked about the new writing format. That means AI scoring. That means, it could be a part of your LSAT score in the future. So it could lead to big changes, even though on the surface, it's just a format switch. I mean that kind of goes back to the point I was making before that, the LSAT and just the whole environment around LSAC is just a lot of change. Moving tests online, going to the Flex, changing the composition of a number of the scored sections, removing Logic Games, now we're changing the Writing. They are changing so much that it's really easy to lose sight of the fact that for 35-plus years this was a static test that law schools trusted. I think that probably makes the LSAT a little bit less important going forward.
Sometimes you change yourself so much, you don't even look like the same person when you come out of it. That can be a really good thing. I don't know if it's a really good thing for the LSAT in this particular instance. So again, another thing we have to watch and see what happens as time goes forward.
Anna: Yeah, I think it's really interesting the notion that it might be scored moving forward. In the beginning of our conversation, you were talking about how you thought that they might replace Logic Games with some different sort of section. So maybe they're doing that on a bit of a delay. For right now, when it's still unscored, I actually kind of like the changes. You said that you weren't a fan.
Dave: Here's my complaint about it. This essay format is not new to standardized testing. We've seen it on the ACT to a great extent before. We've seen it on some AP tests previously. So what they did was move to a different format, but not a new one. What I really would have loved to have seen them do is to lean into the fact that this is a test for prospective law students. And instead of making it a lot of these big moral, social, ethical questions, to make it more of a legal discussion within that same context. Because you can talk about all sorts of things, whether it's copyright and who owns what inside of a legal format or what have you, I would have liked to have seen them say, “hey, let's give people an opportunity to express themselves in a way that they're going to in the future but not have to rely upon them having fore-knowledge of this or prior knowledge.” That was my biggest objection was there's a million legal questions that have social and cultural implications that would be really interesting to talk about. Why not play into the fact that you're the law school test. You talk about being the gold standard, live up to it, make yourself really useful and allow people who do this for a living to see what kind of writing they'd be getting before they have the opportunity to train you in law school.
I don't mind the idea of what they're doing. I think it actually creates a greater degree of complexity. I think it certainly will allow for a greater degree of expression. Like I said about the topic, I could have used a different topic. I have deep concerns about AI scoring. We have seen this on other tests and it sometimes leads to unintended consequences. I'm sure they will start off if and when they finally go to scoring to have an AI-scored and a human-scored. The GMAT did something similar to this before, and what they found was that the humans, over time, kind of morphed into paralleling what the computer scoring did, because the computer scoring was considered to some extent infallible.
The humans adjusted their scoring to what they thought the machine would think, not the other way around. And so AIs are way more advanced than back in those days, but I still think there's certain questions like how creative can you get in these essays? Will there be patterns to them where you can actually crack the code.
So when that starts happening, presuming that it does at some time post-2025, these will be the types of questions that you'll see in the test preparation space where it's like is there a way for us to figure this out and to see exactly what they're doing? And that's what we do with the LSAT right now. Obviously, as you know, we're looking to see what kind of codes that they're using that we can decode and give people tools to figure those things out. The Writing is just going to become another thing like that and I just hope it doesn't become too formulaic. That’s a few of my concerns on here. There are more, but that's a good start.
Anna: Yeah, no, that makes a lot of sense to me. Especially once you get into scoring, like that's its whole own can of worms. The only other standardized test that I've taken that includes a scored writing section is the SAT, which I think probably many of our listeners have taken. Which the scoring of that always bothered me because I really love writing. It's something that I greatly enjoy. And you have to write a crappy essay to do well on the SAT. You need to follow this exact formula. It can't be very interesting. You can't do anything interesting or creative with it.
Scoring, I think there are tons of landmines that they could hit, absolutely. They could make things worse for applicants and less helpful for schools. I think at this stage, before it is scored and the reasons that I like it, although, what you were talking about with subject matter, I think that would make it even better. I do think that this change is going to allow schools to look at the Writing section and have it be a little bit more meaningful.
When I was applying to law school, the advice was you can write pretty much anything in your Writing section, just don't draw a picture and you're probably fine. Because law schools didn't really read them. Which made total sense when it was handwritten, because some people just have such awful handwriting that you can't even parse it. Now that it's typed, I think more schools are looking at it, especially because of generative AI, especially because they're looking at application essays, and not necessarily having the same confidence that they are assessing someone's writing as they might have in the past.
So the LSAT Writing section gives them an opportunity to say, “Okay, this is someone who was writing. No one else could have helped them. It was under specific time constraints. How do they write under those constraints?” When I was Interim Assistant Dean of Enrollment Management at Syracuse, I would read those essays for that purpose. I would read the LSAT essays in order to gauge writing. But beyond just the question of can this person form coherent sentences and write with some level of clarity, you couldn't get a whole lot from sort of their argumentation abilities because it was so basic. All you could do really was parrot back, “These are the facts that they told me and that's why I think XYZ.” There wasn't a whole lot of differentiation. Certainly, it was helpful in terms of determining, okay, can this person communicate well in written English. But it was not particularly helpful in determining is this person a strong written communicator versus an adequate written communicator. I think that to some degree could be helped by this, we'll see it ends up turning out.
But I do think that law schools are quietly starting to read LSAT Writing sections much more than they used to. And I think that this will probably increase that, That's my speculation at this point. We'll see.
Dave: I think it's a good guess for whatever that's worth, because I agree with you on this. I think, especially when it first comes out, law schools will want to just look at it, see what kind of outcome is it producing. But now, these days, it really is hard to know. You mentioned generative AI and personal statements. And you're like, did the person write this or did they not? It's really hard to tell. Well, I can go check your LSAT Writing and see whether it matches it stylistically, tonally, what have you. So I think it's going to become more important.
And LSAC has said, look, they're getting this from the law schools. The law schools value the writing portion and they want to see it highlighted somehow. They probably knew that it had been too formulaic, and I think that it had been. It's a much easier essay to write the current one than it will be to write the future one. It's going to take more work, more thought, and that's why they gave you the 15 minutes up front, is because they realize that. So you still have the same amount of writing time. That should lead to better essays and hopefully creativity is at least in the first year rewarded and not penalized even so if they start to score it at some point. So, I do think they at the least will glance at it, even if they only read one paragraph of it. And it doesn't even have to be the first paragraph. They might just say, let me just glance at this and see if it's matching up. It becomes a double-check tool. And that by itself is useful. And then if you can see further insights into somebody's ability, then it's even more useful.
Anna: Great on all counts. So I think one piece of advice that I would give to folks who are going to be taking the LSAT in August 2024 or later would be to put some real effort at least in the moment. I still don't think you probably should be doing a bunch of prep beforehand, but in the moment of taking your LSAT Writing section, I think you should really give it your best effort and really try to write a strong essay. Even though the advice 10-plus years ago might have been, “write literally anything, just don't draw a picture.”
What other advice? This has been a very, very broad conversation, and I think to some degree, the advice is going to be individual based on who you are, what you're good at, and how you look at a test like the LSAT. But what sort of general advice can our listeners walk away from this podcast episode with? In terms of all these changes that we've been talking about.
Dave: Well, the first one I'll actually give will connect back to that LSAT Writing. Because some people have a current LSAT Writing and they wonder if they have to take it again. You don't have to take it again. And some people will wonder whether or not having a new one is going to be more beneficial than the old one. I think early on, it's a wash. There'll be so many of the old ones out there that it won't actually make a difference.
But probably the broader takeaway out of all of this is that we're in a period of uncertainty. Change is happening and has been happening over the last five years in such a way that for law schools, it slows them down, it gives them pause. People have a very specific cadence in mind about when to apply, when they will get decisions, when the financial aid will come through, when they have to make a final decision. And as we've seen in the last couple of years, that's been getting pushed back. All that's happening here with the LSAT is not helping that. It's not making that timeline speed up. It will continue to keep going slowly. So the expectation as an applicant would be it's going to take some time, these law schools need to figure things out. They do not want to jump in quickly. Some schools did that before and got burned a little bit. Each of those changes could be good or bad for you as an applicant. It's almost like an individual case.
There's no kind of generalization that says, “this will help everybody.” You have to look at your own strengths and weaknesses and figure out, “Should I take a Logic Games LSAT, or should I take one that doesn't have Logic Games? These types of things are going to affect you on a personal level, and only you will know what is ultimately the best for you. So you're going to have to ride the wave of uncertainty here, and it will be a slow wave because of the way this is affecting law schools. And that's probably the best overall nonspecific advice that I can give. Obviously with individuals, it'll change radically because some people have very specific courses of action they should take and hopefully they have a really good grasp with that sooner as opposed to later.
Anna: That’s great advice. It pains me a little bit to already be starting to predict a slow cycle next cycle also. We've been talking about how this current 2023-2024 cycle has been the slowest cycle ever. We've been talking about that from pretty early on, just because there were so many factors that have changed. And I've been hoping that it would remain the slowest cycle ever. I would prefer, for applicants’ sake, for things not to continue to slow down, but I think you're right. I think it's a very, very strong possibility, especially given the last time the LSAT had a significant change. Law schools thought that things were going to stay basically the same. We were told by LSAC that things were going to stay basically the same. And the law schools that moved early and acted in accordance with those expectations were burned. As you said, they ended up not being able to increase as much as other schools were able to increase. In a lot of cases, they ended up over-enrolling so much so that across the board, it was ten percent up from the previous year's class just across all law schools.
So I do think that schools have a reason to move more slowly in the face of this type of uncertainty, just based on the very recent past. So expecting a slow cycle is probably good advice, as much as I wish that we would be dead wrong about this.
My only other piece of advice that I would add is that obviously numbers are important. The LSAT is going to continue to be important. Your undergraduate GPA is going to continue to be important. But now more than ever, I think applicants should really not be discounting the importance of their other application materials, of their sort of "soft" factors. I don't just mean putting thought into your personal statement and your other essays and things like that. I also mean more broadly, sort of, what do you have on your resume? What have you been up to? What work have you done? What did you do during college? I think all of these things are going to start to have at least a little bit more weight.
I think this sort of is just changing the overall tenor of what law school admissions and law school admissions advice looks like. For a long time, as we talked about earlier on in this episode, for a long time my advice has always been first thing you should know about law school admissions is that it's very numbers-focused. Much more numbers-focused than undergraduate admissions, as I was saying, much more numbers-focused than any other program that I'm aware of. And I do think that's lessening slowly to some degree.
As you mentioned, law schools, legal education moves slowly. Change does not happen overnight at the vast majority of law schools, things move slowly. So this is not a black and white thing. It's not a 180 that law schools are doing, but I do think it's increasing the importance of these other elements apart from the numbers. So definitely focusing on those other elements of your application and not just discounting them because you have great numbers, I think would be my other piece of advice.
Dave: That's great advice too, because it's so easy to get focused on numbers, and it's been that way for so long, and it's black and white, your GPA, your LSAT score, once it's on the paper, it's a hard number that you can look at. There's a reason the rest of this is called “softs.” But they are more important. And even U.S. News is telling people that they're more important by decreasing the emphasis on LSAT and GPA. So more than ever, what I always tell people when it comes to admissions and certainly, you put it in the Law School Admissions Bible, is take your time, work through it. This is not a race that is a hundred-meter sprint. This is more of a marathon, you want to have the best possible application, not the fastest possible application.
Anna: Yep. I think that is an excellent point to end on. Thank you, Dave, for your time. I greatly, greatly appreciate it. I know you're a busy person, and you have been paying attention to the LSAT and you have been an LSAT expert for a very long time. And having that longer term perspective, as we are in this period of great change, I think is incredibly valuable. So thanks for taking the time and for sharing some of your wisdom with our listeners.