Here’s the latest paper published on the comparison of decoder-only LLMs (ChatGPT, Claude, and Gemini) on how well they perform sentiment analysis for short text across multiple languages. We compared how accurate they were when asked to evaluate the text in its original language versus when that text was translated to English. Additionally, we compared the decoder-only LLMs to encoder-only LLMs (such as BERT and its variants), Recurrent Neural Networks, and lexicon-based sentiment analysis methods (such as VADER) to see which produced the most accurate results.

 We found that decoder-only LLMs achieved the highest accuracy across all sentiment analysis methods when working with the original language data. The only exception was with the French data, where an RNN was the most accurate. Among the three decoder-only LLMs, ChatGPT had the highest accuracy in four of the seven languages, Claude in two, and Gemini, which ranked second in six of the seven languages.

Link to full paper: https://jsomer.org/index.php/pub/article/view/38

These are a few samples of one of the maps produced by my thesis in Chapter 7. I could produce them within the minute timeframe as the text data from the tweets was being processed. These maps were from March 17th, 2023, where there was a lot of geopolitical action surrounding North Korea testing missiles.

Map 1:

Map 2:

Map 3:

The cluster of tags off the coast of West Africa are errors, either entities that would incorrectly classified as places or did not have an identifiable latitude and longitude. Below is the key for the maps.

Thanks for reading and please contact me with any questions!

This is my first blog post, I will try an update this more frequently in the future with more projects I have been working on.

Test Post