ChatGPT's Tips for Preparing for a Practice MBE Exam

It's the end of January, so this is the time when people preparing to take the February 2023 bar exam are likely getting ready to take a simulated MBE exam offered by one of the commercial bar review companies.

Taking a full, 200-question milestone exam this well will maximize time after the practice exam and before the bar exam to identify and remediate their weaknesses.

On another note, some of you may have been following the artificial intelligence chatbot ChatGPT and how it was able to pass a “main part” of the bar exam. The news reports are a bit misleading because, as you know, there is no minimum passing score on the MBE, at least on the Uniform Bar Exam. An examinee’s scaled score on the multiple-choice section is added to the examinee’s scaled score on the written portion of the exam. The combined scaled score is what’s used to determine whether one passes or fails the bar exam.

Still, researchers found that ChatGPT was able to answer correctly 50.3% of the Evidence and Torts MBE questions during the research, which is considerably better than pure guessing but not where examinees want to be for the bar exam. On the bar exam, examinees will want to strive for answering 65% of the Evidence MBE questions correctly and 75% of the Torts MBE questions, especially to compensate for the harder MBE subjects like Contracts and Real Property, where scores of 55% to 60% can actually be comparatively solid.

In the law school context, my alma mater recently used ChatGPT to generate answers on four real exams at the University of Minnesota Law School. The professors for those courses then blindly graded those AI-generated responses as part of their regular grading processes for each class. The ChatGPT responses to more than 95 multiple-choice questions and 12 essay questions performed the same as a C+ student, achieving a low but passing grade in all four courses.

It'll be interesting to see how ChatGPT wll perform on one of these 200-question simulated exams offered by one of the commercial bar review companies. That’s on my to-do list, whenever I can make progress on my to-do list.

This morning, however, I did ask ChatGPT how to prepare for the BARBRI Simulated MBE exam that many of you will taking this week. Some of the tips ChatGPT provided were actually tips on what to do after the exam.

Still, it's interesting to examine what ChatGPT "said." Here are ChatGPT’s responses and my thoughts about the AI-generated tips:

ChatGPT’s response: “1. Review the material covered on the MBE: Make sure you have a solid understanding of the legal concepts and principles that will be tested on the exam.”

My comments: Even if you’re not fully prepared to take the BARBRI Simulated MBE, go ahead and take it. You will never feel completely prepared to take the exam. But, again, the benefit of taking the exam isn't necessarily to assess how much you’ve learn up to that point. Rather, the benefit of the simulated exam is to be able to use the information from the resulting performance report to help guide and steer your preparation for the next few weeks.

ChatGPT’s response: “2. Take practice exams: Practice taking simulated MBE exams under timed conditions to get a sense of the pacing and format of the test.”

My comments: The BARBRI Simulated Exam is just the first step of taking practice MBE exams under timed, test-like conditions. You will want to incorporate these other practice exams into your schedule: Practice Exams 1 and 2 (100 Questions in the BARBRI MCQ Bank); the 100-question BARBRI Refresher; the 100-question BARBRI MBE Final Prep (these are all released NCBE questions); and the BARBRI Half Day and Full Day examinations.

ChatGPT’s response: “3. Analyze your mistakes: After taking a practice exam, review the questions you got wrong and understand why you made those mistakes.”

My comments: ChatGPT’s suggestion is incomplete. Yes, you should review the questions you answered incorrectly. But you should also review the questions you answered correctly. In other words, review all the questions—whether you answered them right or wrong. First, you might have answered a question correctly but for the wrong reason. Second, you can actually learn more information by fully understanding why the test drafters created the incorrect answer choices.

ChatGPT’s response: “4. Make flashcards: Make flashcards of key legal terms and definitions to help you memorize important information.”

My comments: Creating flash cards highlighting key points is a good strategy for those who have mastered the “big picture” but need to refine their knowledge of nuances. But, again, ChatGPT’s suggestion is incomplete. Here are other techniques may help you:

• Creating mind maps/charts/flowcharts highlighting key issues for each subject. This is good for visual learners.

• Recording yourself reciting information about different subjects and listen to your recordings. This is good for learners who retain what they hear.

• Giving yourself a lecture on a subject using your notes. This means that you need to understand the subject and be able to explain it clearly. This is also good for learners who retain what they hear.

• Enlisting friends or family to listen to you explain a particular subject. This is helpful because explaining things to an audience means that you cannot hide from any aspects of the topic that you do not yet fully understand.

• Writing key information out multiple times. This can be a useful technique for those who find that repetition is key to mastering material.

• Reviewing problem areas in the bar review course's outline materials. Don’t review the whole outline but instead the specific subtopics that you are struggling with.

ChatGPT’s response: “5. Get familiar with the format and structure of the test: The MBE is a multiple-choice test, so make sure you are comfortable with that format and know how to approach each question.”

My comments: I don’t disagree with this.

ChatGPT’s response: “6. Get enough rest and stay healthy: The MBE is a long and challenging exam, so make sure you are well-rested and in good physical and mental health.”

My comments: Absolutely. Even though the BARBRI Simulated MBE is a practice exam, treat it like the actual exam. That means getting rest beforehand—like you should get before the February bar exam—eating the same foods for breakfast and lunch that you anticipate eating on the days of the bar exam, and even wearing the same clothes you expect to wear in the actual testing location.