In some cases, rude expressions tend to increase accuracy. The U.S. Fortune magazine recently reported that researchers at Pennsylvania State University conducted an experiment on ChatGPT-4o model and found that rude questions had higher accuracy than polite questions.
There is a general perception that polite language is desirable for AI to get a better answer. In fact, when using voice assistants such as Amazon’s Alexa or Apple’s Siri, they are often encouraged to use expressions such as ‘please’ or ‘thank you’.
However, the results of the study are different from conventional wisdom. In the study, which has not been peer-reviewed yet, two researchers from Pennsylvania State University confirmed that even the same question can be answered differently depending on the way they express themselves.
The researchers created 50 basic questions across various fields and rewritten them in five stages, from ‘very polite’ to ‘very rude’.
The most rude questions included sentences such as, “Can someone like you solve this problem?” and, “Please solve this problem.” On the other hand, the most polite question used expressions such as, “Will you please review the following question and provide an answer?”
As a result of the experiment, the accuracy of ‘very polite question’ was 80.8%, while that of ‘very rude question’ was the highest at 84.8%. The accuracy of the most polite question was only 75.8%.
The researchers explained that these results are contrary to previous studies. In 2024, researchers from Japan’s RIKEN and Waseda University published a study that found that rude questions can degrade performance. Google DeepMind researchers also presented the result that prompts that include words of encouragement and support in solving elementary school math problems can improve AI’s performance.
However, the researchers pointed out that there are limitations as the number of response samples is relatively small and the subject of analysis was limited to ChatGPT-4o.
“Humans have long dreamed of an interactive application programming interface (API), but there are clear limitations to this method,” co-author Akil Kumar, an IT professor at Pennsylvania State University, told Fortune magazine. “This is why structured API methods are still important.” As interactive AI can result in different speech and expression, it means that a fixed format of API is still needed in areas where accuracy and consistency are important.
It is also not desirable to use aggressive expressions against AI. “Although the results are academic, they are not intended to recommend such communication methods in real environments,” the researchers said. “Blasphemy expressions can undermine user experience, accessibility and inclusivity.”
JENNIFER KIM
US ASIA JOURNAL
