Navigating User Behavior toward Personalized Multimodal Generation
Summary
NaviGen is a novel approach addressing the misalignment between user intent and modern AIGC pipelines in personalized multimodal content generation. It tackles the challenge of users rarely articulating visual details by turning interaction history into executable instructions. NaviGen employs a dual identifier, coupling collaborative and textual codes, as a behavioral substrate and semantic bridge within a single token stream. Its two-stage SFT+RL pipeline first distills preference reasoning and instruction writing from evolutionarily searched supervision, then aligns generation with user intent using hierarchical and self-consistent rewards. Experiments across product, game, and short-video domains demonstrate NaviGen's ability to improve personalized image and video generation, strengthen next-item prediction, and yield more specific, relevant, and visually generatable instructions.
Key takeaway
For AI Scientists developing personalized content generation systems, NaviGen offers a robust framework to bridge the gap between user intent and AIGC output. You should consider implementing a dual-identifier representation and a two-stage SFT+RL pipeline to encode user behavior effectively and acquire instruction-writing capabilities. This approach can significantly improve the specificity and relevance of generated content across diverse domains like product, game, and short-video applications.
Key insights
NaviGen translates user history into executable instructions for personalized multimodal generation via dual codes and a two-stage SFT+RL pipeline.
Principles
- Encode user behavior legibly for language reasoning.
- Models must acquire instruction-writing skill.
- Align generation with user intent.
Method
NaviGen represents items with dual collaborative and textual codes. A two-stage SFT+RL pipeline distills preference reasoning and instruction writing, then aligns generation with user intent using hierarchical and self-consistent rewards.
In practice
- Improve personalized image generation.
- Enhance next-item prediction in games.
- Generate specific video instructions.
Topics
- Personalized Generation
- Multimodal AIGC
- User Behavior Modeling
- NaviGen
- SFT+RL Pipeline
- Instruction Writing
Code references
Best for: Research Scientist, AI Scientist, Machine Learning Engineer
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.