Community detection is an important task to reveal hidden structures of real-world complex networks which are vary over time. Most of the existing works on the dynamic community detection assumes the sparse connectivity between communities and supposes that the number of nodes and communities in different snapshots is constant. In this work, a probabilistic overlapping community detection method called PODCD is proposed that considers the task of detecting communities as a non-negative matrix factorization problem. The proposed method considers the more likely assumption of dense connections between communities and utilizes a probabilistic model to control the dynamics of community structure. The proposed method uses the block coordinate decent method to solve the objective function of the matrix factorization model. This solver estimates non-negative latent factor to speeds up the computation of gradients. We demonstrate the efficiency of the proposed method by performing experiments on several synthetic and real-world dynamic networks. The obtained results reveal that the proposed method outperforms the earlier algorithms on evolving networks in terms of well-known evaluation criteria.