In this talk, I will introduce a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization. The proposed policy regularization induces a sparse and multi-modal optimal policy distribution of a sparse MDP. The optimality condition of a sparse MDP and sparse value iteration will be discussed and compared to existing methods. The convergence and optimality of sparse value iteration will be presented. Interestingly, it can be shown that the performance error of a sparse MDP has a constant bound, while the error of a soft MDP increases logarithmically with respect to the number of actions. In experiments, sparse MDPs are applied to reinforcement learning problems and outperform existing methods in terms of the convergence speed and performance.

If time permits, I will also discuss our group’s recent work in deep learning, including Text2Action and nested sparse networks.