A new study that came out on Wednesday says that AI can currently do better on the bar exam than most law school graduates. If you want to be a lawyer in the United States, you have to pass the two-day bar exam.

This week, OpenAI, which is backed by Microsoft, released a new AI model called GPT-4. In an experiment, two law professors and two Casetext staff members gave it the bar exam and gave it a score of 297.

According to the researchers, GPT-4 is in the 90th percentile of actual exam takers, which is sufficient to practise law in most states.

The bar exam gauges your level of knowledge and critical thinking. The performance exams, essays, and multiple-choice questions all simulate the process of conducting legal work.

Massive language models, according to the researchers, are capable of handling challenging tasks requiring in-depth legal knowledge, reading comprehension, and writing ability. "In almost every American jurisdiction, human attorneys are held to a standard.

Also Read: Apple blocks update to email app with ChatGPT tech

Less than four months ago, two of the same researchers who are now working on OpenAI's current huge language model, ChatGPT, reached a different conclusion. This demonstrates how rapidly technology is evolving.

When compared to ChatGPT's 50% accuracy rate, the more modern GPT-4 exceeded the average human test-taker by more than 7%, successfully answering more than 76% of the multiple-choice questions on the bar exam.

The National Conference of Bar Examiners, which develops the multiple-choice portion of the exam, asserted in a statement issued on Wednesday that attorneys have specific skills that they have honed through education and practice that "AI cannot presently match."

Also Read: Google makes its ChatGPT rival AI, "Bard," available to beta testers

According to research co-author and professor Daniel Martin Katz, the ability of GPT-4 to produce typically germane and logical essay and performance test replies astounded him the most.

I heard so many people say, "Well, it might get the multiple choice, but it will never get the essays," said Katz.

Although AI performed well on the SAT, GRE, and other standardised tests, the bar exam has garnered more attention. On Tuesday, OpenAI boasted about how well the most recent model performed.

A tutor for the bar exam named Sean Silverman attributed the emphasis on the bar exam to its well-known difficulty. 78% of test takers this year passed the solicitor licence exam on their first try after completing three years of law school.

If AI can pass a high school exam like the SAT, as Silverman put it, "rather than the exams to become a barrister," people will be less astonished.