When Models Meet Users: An Empirical Study of Perceptions of General LLMs and Multimodal LLMs on Hugging Face

2026-03-16 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, extended

Summary

An empirical study analyzed 662 discussion threads from 38 general and multimodal large language models (GLLMs and MLLMs) on Hugging Face to understand user perceptions. Researchers manually annotated discussions from 21 GLLMs and 17 MLLMs, developing a three-level taxonomy of concerns. Key findings reveal that access barriers, particularly for the Llama model family, are a primary obstacle, with 69 GLLM threads and 16 MLLM threads discussing access permissions. Generation quality is a core evaluation dimension, with MLLM users showing higher sensitivity (46 threads) than GLLM users (7 threads). Model invocation and deployment present significant challenges, especially for MLLMs (75 threads vs. 46 for GLLMs), reflecting lower ecosystem maturity. Insufficient documentation, the technical difficulty of fine-tuning (21 GLLM, 28 MLLM threads), and GPU/memory limitations (13 GLLM, 14 MLLM threads) are also prominent issues, driving the adoption of quantization techniques.

Key takeaway

For AI Engineers and ML Scientists deploying or developing LLMs, you should prioritize addressing user pain points identified in this study. Focus on providing clear, official quantized model versions to mitigate GPU memory constraints and standardize deployment frameworks to simplify integration. Enhance documentation with practical examples and advanced task guidance, as these directly reduce learning curves and improve adoption, especially for complex multimodal models.

Key insights

User feedback on Hugging Face reveals LLM ecosystem maturity gaps, highlighting access, quality, and deployment as critical pain points.

Principles

• LLM access barriers hinder adoption, especially for gated models like Llama.
• Multimodal LLMs face higher generation quality and deployment complexity.
• Documentation quality directly impacts developer learning costs.

Method

The study collected 662 discussion threads from 38 GLLMs and MLLMs on Hugging Face, manually annotating them using a three-level taxonomy to characterize user concerns and sentiments.

In practice

• Prioritize official quantized model versions to reduce hardware barriers.
• Standardize deployment frameworks for consistent environments.
• Enhance documentation with clear examples and advanced task guidance.

Topics

• LLM User Experience
• Multimodal LLMs
• Hugging Face Ecosystem
• Model Deployment
• Quantization Techniques
• Access Control

Code references

• ggml-org/ggml

Best for: NLP Engineer, Computer Vision Engineer, Research Scientist, Machine Learning Engineer, AI Engineer, AI Scientist

Related on AIssential

• LLM Architecture in 2026: Agent Harnesses, Hybrid Models, and Why Implementation Does Not Lie — Implementation-first analysis of LLM architectures and agent systems reveals evolving training, inference, and context management strategies.
• Beyond Isolated Behaviors: Hierarchical User Modeling for LLM Personalization — LLM personalization benefits from a hierarchical user model based on practices, habitus, and fields.
• What Do Models Still Suck At? - Peter Gostev, Arena.ai, BullshitBench — LLMs still struggle with identifying nonsense and consistently meeting user expectations in complex, real-world tasks.
• Mapping the Design Space of User Experience for Computer Use Agents — Understanding user experience is crucial for designing effective LLM-based computer use agents.
• The Model You Love Is Probably Just the One You Use — Developer preference for LLMs is often driven by familiarity and integration ease, not objective superiority.
• Persona Cartography: Charting Language Model Personality Traits in Weight Space — LLM personas can be precisely controlled and measured as positions in a behavioral trait space using the OCEAN framework and low-rank adapters.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.