Diversity-Constrained Long-Term Policy Learning for Exposure Balancing in Recommenders

Abstract

The rapid proliferation of digital content platforms has cemented recommender systems as the primary mediators of information consumption. However, the prevailing paradigm of maximizing immediate user engagement often engenders systemic exposure bias, wherein a small subset of popular items receives disproportionate visibility while niche or newly introduced items remain perpetually underexplored. To address this pervasive algorithmic inequity, this paper introduces a novel framework termed Diversity-Constrained Long-Term Policy Learning. By formulating the recommendation process as a constrained Markov Decision Process, the proposed framework explicitly balances the maximization of long-term user satisfaction with rigorous exposure fairness across the item catalog. The methodology integrates a customized policy gradient optimization technique that continuously adjusts the recommendation distribution based on historical exposure trajectories, thereby preventing the collapse of item diversity. Furthermore, we introduce an adaptive baseline architecture that dynamically scales the diversity constraints according to user engagement levels, mitigating the traditional trade-off between recommendation accuracy and fairness. Comprehensive theoretical analysis and subsequent algorithmic formulations provide a robust foundation for understanding the dynamics of long-term exposure. Empirical evaluations conducted on multiple large-scale interaction datasets demonstrate that the proposed framework significantly enhances catalog coverage and exposure equity while maintaining highly competitive cumulative reward outcomes compared to state-of-the-art reinforcement learning baselines. The findings offer profound implications for the design of sustainable, equitable, and diverse digital content ecosystems.