Sentiment analysis on social media, then and now

What the original approach did, and why it stopped being the default

A decade ago, sentiment analysis on social media posts — the original version of this article used it to gauge public perception of political candidates from tweets — meant a pipeline built from parts that had to be assembled by hand: tokenize the text, strip stopwords, build a bag-of-words or TF-IDF feature vector, and feed that into a classifier like Naive Bayes or a linear SVM trained on a labeled dataset of positive and negative examples.

That pipeline worked, in the narrow sense that it produced a number. It also had real, well-known limits: it had no notion of context (negation, sarcasm, and mixed sentiment within one sentence routinely confused it), it needed a labeled training set specific to the domain and language it was applied to, and it was blind to anything the training data didn’t cover — new slang, code-switching, a candidate’s name used sarcastically.

The current default: a language model, prompted or fine-tuned

The equivalent task today is far more likely to run through a language model — either a general-purpose one used zero-shot with a prompt, or a smaller model fine-tuned for the specific classification. The zero-shot version needs no labeled training set at all:

from anthropic import Anthropic

client = Anthropic()

def classify_sentiment(text: str) -> str:
    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=10,
        messages=[{
            "role": "user",
            "content": f'Classify the sentiment of this social media post as '
                       f'positive, negative, or neutral. Respond with one word.\n\n'
                       f'Post: "{text}"',
        }],
    )
    return response.content[0].text.strip().lower()

This handles negation, sarcasm, and mixed sentiment far better than a bag-of-words model, because it’s reasoning over the whole sentence’s meaning rather than counting the presence of individual words stripped of their surrounding context. It also needs no domain-specific training data to get started — the model’s general language understanding carries over.

What this costs that the old approach didn’t

None of this is free, and the honest comparison has to include the costs that a classical model doesn’t have:

• Latency and cost per classification. A classical model scores in microseconds on commodity hardware; an API call to a language model adds real per-request latency and a per-token cost. At the scale of classifying millions of social media posts, that cost difference is not trivial, and it should be part of the decision, not an afterthought.
• Determinism. A classical model gives the same output for the same input every time. A language model, especially prompted rather than fine-tuned, can drift — a prompt tweak, a model version update, or sampling temperature above zero can all change the answer for the same input, which matters if the classification feeds something that needs to be reproducible or auditable.
• Explainability. A linear model’s weights tell you, directly, which words pushed a classification toward positive or negative. A language model’s reasoning is opaque unless you explicitly ask it to explain itself, which adds tokens, cost, and still isn’t a guaranteed faithful account of what actually drove the answer.

Where a classical model still wins

For a well-defined, high-volume, narrow classification task — is this support ticket urgent, is this product review positive or negative, is this transaction likely fraudulent — a small classical model, or a small fine-tuned transformer distilled for the task, is very often still the right engineering choice: cheaper per inference by orders of magnitude, fast enough to run inline on every request rather than batched, and simple enough to audit when something goes wrong. Reaching for a large general model on every classification task, out of habit rather than need, is a real cost that compounds at scale.

The practical split most teams land on: use a language model to bootstrap and label a dataset quickly, or for early exploration where volume is low and you don’t yet know what you’re looking for; move to a smaller, cheaper, fine-tuned model once the task is well-defined and running at meaningful volume in production.

Evaluation matters more than the choice of model

Whichever approach is used, the discipline the original article’s approach mostly skipped is rigorous evaluation: a held-out, human-labeled test set that neither model saw during development, checked against the standard classification metrics (precision, recall, F1) per class, not just overall accuracy — which can look deceptively good on an imbalanced dataset where most posts are neutral. A model that’s 90% accurate but only because 90% of the data is neutral and it labels everything neutral is not a working sentiment classifier, and this is exactly the kind of failure a single accuracy number hides and a per-class breakdown reveals immediately.

What actually changed

The technique moved from hand-assembled linguistic features and a domain-specific classifier to a general-purpose language model that needs no training data to start working, at the cost of higher per-inference cost, less determinism, and less transparency into why it decided what it decided. Neither approach is obsolete in an absolute sense — the language model is usually the better starting point today, and the classical model is often still the better production choice once the task and its volume are well understood.

---

Originally published in 2015 and updated for 2026.