Abstract: To deal with the problem of modeling pixel-pair relationship for foreground segmentation in video sequences, we propose a multi-layer Markov random field (MRF) model fusing entropy information. Pixel model and smooth model are encoded into the Markov random field framework to update the weights of spatio-temporal constraints. The algorithm of loopy belief propagation makes the global energy optimization more effective. Experimental results for different video sequences show our developed method has a better veracity of segmentation results.