Multimodal NLP for Embedded Psychological Text Analysis

Accurate psychological text analysis on embedded devices is essential for privacy-preserving mental health monitoring; however, deploying transformer-based models in such constrained environments remains challenging due to limited memory, computational power, and energy budgets. We present a multimodal framework that integrates a lightweight Transformer (DistilBERT) with a CNN-based image branch (ResNet-50) accelerated by the Hailo-8L edge AI board. Text inputs are processed both as sequences and as two-dimensional image-like representations, enabling robust handling of noisy, informal text containing emojis and misspellings. Evaluations on the Depression Twitter dataset and the SST-2 benchmark demonstrate that the fusion model consistently outperforms text-only and image-only baselines, achieving up to 91.6% accuracy while remaining feasible for deployment on a Raspberry Pi 5. Overall, the proposed design offers a practical balance between efficiency and accuracy, enabling real-world, privacy-aware psychological monitoring directly on personal embedded devices.