Fast growth of video data on the Internet requires managing these video data. In the last decade, Content-Based Video Retrieval (CBVR) became a considerable research interest to handle the large amounts of collecting video media on the Internet. Due to concerning a large ratio of the videos on the Internet to humans, human action retrieval is presented as a new topic in CBVR domain. In this paper, we seek to improve the current state-of-the-art retrieval algorithms for CBVR by using the statistical information. The statistical information is utilized to represent the video by a vector with m units instead of local points of m×n units and creating a histogram of Bag of Words and also, it decreases the complexity significantly. Furthermore, each vector is used to compare the videos and to find the similar videos to the query one instead of histogram of Bag of Words. The experimental results on KTH dataset illustrated that in contrast to the Bag-of-Words model and its various parameters, our method can perform better.