AI bot ChatGPT fails to clear India’s Civil Services Prelims.
What is AI ChatGPT?
AI ChatGPT, Chat Generative Pre-trained Transformer, is an Open AI chatbot. This technology has revolutionised the Internet. It is a deep learning model that has been trained on a large amount of text data, making it capable of performing a wide range of Natural Language Processing tasks such as text generation, machine translation, sentiment analysis, and chatbot development. In the training process, the model learns the patterns and relationships between words and phrases in the text data, allowing it to generate new texts that are similar to the training data.
A Bengaluru-based organization, Analytics India Magazine (AIM), tried to check on the proficiency of chatbots. According to an analysis report, ChatGPT failed to clear the UPSC prelims. It could answer only 54 questions correctly, out of a total of 100 questions. It tested the chatbot by giving it questions from Set A of the 2022 UPSC Prelims. The magazine said that the AI bot didn’t meet the 87.54% cut-off for the general category UPSC aspirants. This was the 2021 cut-off, the magazine said.
India’s civil service exam is considered one of the toughest examinations in the world. ChatGPT was not able to answer questions varied from history to science to economics to current affairs to geography among other subjects. When it was asked from ChatGPT whether it was possible for it to clear the UPSC prelims, the chatbot said that though it has knowledge from various subject areas, clearing UPSC also requires ‘critical and logical thinking’ along with time management.
The chatbot abstained from giving a definite yes or no as an answer to questions. While in some questions, ChatGPT created its own options. In MCQs, where candidates have four options, ChatGPT created a fifth option ‘E’! The Analytics India Magazine said that ChatGPT’s knowledge is limited to 2021. Hence, it was unable to answer questions related to current affairs after 2021. However, several questions related to geography, which are not time-bound were also answered incorrectly by the chatbot.
ChatGPT may not be able to clear the UPSC but it aced the Wharton exam. It was also able to crack the law exams at the University of Minnesota. It was launched in November 2022 and has gained immense popularity and attention. ChatGPT in recent days cleared some other most difficult exams including the US Medical Licensing Exam, the Master of Business Administration (MBA) programme at the University of Pennsylvania’s Wharton School and many other prestigious MBA exams. It also cleared Google Coding Interview for Level 3 Engineers. However, it terribly failed to qualify for the UPSC Prelims.
LIMITATIONS OF CHATGPT
Despite its many advantages, GPT has its limitations too. Some of the key limitations are:
- Biases: ChatGPT is trained on a large corpus of text data sets, which may contain biases and prejudices. This can result in the model generating biased text, which can be irrelevant, harmful and offensive.
- Lack of Context: GPT is trained on text data and does not have access to any other external information. This means that it may not be able to provide reference to context information, making it unsuitable for certain NLP tasks.
- Lack of Understanding: GPT is not capable of understanding and correlating the meaning of the text, it simply generates text based on patterns and relationships learned from the training data.
ChatGPT is a cutting-edge language model that offers a wide range of benefits and assistance to users. With its ability to generate high-quality, contextually appropriate text, Chat GPT is an ideal tool for businesses and organizations looking to communicate with their consumers and clients in a more personal and human way. Additionally, the model’s ability to learn and improve over time makes it a highly adaptable tool that will continue to be of immense value for years to come.
To Read more on Artificial Intelligence, check the following post https://legalresearchandanalysis.com/artificial-intelligence-and-its-impact-on-jobs-and-society/
About the Author
