Personality Classification: Introvert vs Extrovert from Text using NLP

2026-03-20 · Source: NLP on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, long

Summary

A project utilized Natural Language Processing (NLP) techniques to classify personality traits as Introvert or Extrovert from text data, aiming to overcome the time-consuming nature of traditional questionnaires like MBTI. The system uses the MBTI Personality Type Dataset from Kaggle, specifically the mbti_1.csv file, which contains 8,675 samples. Exploratory Data Analysis revealed a significant class imbalance, with approximately 6,700 Introvert samples and 2,000 Extrovert samples. The text data, averaging 7,235 characters and 1,200-1,500 words, underwent preprocessing including lowercasing, removal of links, numbers, and special symbols, and label encoding (Introvert=1, Extrovert=0). An LSTM-based Deep Learning Sequential Model was developed with an Embedding Layer (192,000 parameters), an LSTM Layer (24,832 parameters), a Dropout Layer (0 parameters), and a Dense Output Layer (65 parameters), totaling 216,897 parameters. The model was trained with EarlyStopping and class weights to address imbalance, achieving an overall accuracy of 79% but showing a strong bias towards predicting Class 1 (Introvert).

Key takeaway

For AI Engineers developing text-based personality classifiers, recognize that class imbalance is a critical challenge that can lead to biased models despite high overall accuracy. You should prioritize increasing data for minority classes, employing techniques like class weighting, and optimizing decision thresholds based on metrics like F1-score to ensure balanced performance across all personality types. This approach will yield more robust and equitable classification systems.

Key insights

NLP can classify personality traits from text, but data imbalance significantly impacts model performance.

Principles

• Class imbalance biases models.
• Text length alone is not a personality differentiator.
• F1-score is better for imbalanced data.

Method

The method involves text preprocessing, tokenization, padding, and training an LSTM model with class weights and EarlyStopping for binary personality classification from text.

In practice

• Use class_weight for imbalanced datasets.
• Adjust decision thresholds to optimize F1-score.
• Increase minority class data for better generalization.

Topics

• Personality Classification
• Natural Language Processing
• Long Short-Term Memory
• Class Imbalance
• Text Classification

Best for: AI Engineer, Machine Learning Engineer, Data Scientist

Related on AIssential

• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.
• Automatic Evaluation of Enem Essays: A Comparative Analysis between Feature Engineering and Transformers — Regression formulations are generally more suitable for ordinal essay scoring than multiclass classification.
• Clasificación de Sentimientos en Reseñas de Películas con Naive Bayes en Python — Naive Bayes, with proper preprocessing, offers robust and interpretable sentiment classification.
• Indigenous Letters to Brazil: Multi-Label Classification — Multi-label classification of indigenous letters presents challenges due to corpus imbalance and semantic overlap.
• From Logistic Regression to GPT-2: Building a Complete Spam Detection & Sentiment Analysis Pipeline — For imbalanced text classification, prioritize F1-score and Precision-Recall AUC over accuracy and ROC-AUC.
• Discovering Semantic Latent Structures in Psychological Scales: A Response-Free Pathway to Efficient Simplification — Semantic analysis of questionnaire items can efficiently simplify psychological scales without requiring respondent data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NLP on Medium.