Fast growth of multimedia data (e.g. videos) on the web makes some challenges on regular searching methods. To this end, Content-Based Video Retrieval (CBVR) was introduced as a considerable research interest for managing the collected videos' search on the Internet. Furthermore, due to relating most of these videos to humans, human action retrieval is considered as a new topic in CBVR. In this paper, we seek to improve the accuracy of state-of-the-art CBVR retrieval algorithms with minor computational cost. In this method, local feature points of each video are extracted and the moving directions and scales of the included action are calculated using the points' gradient. The point's gradients on different axis are concatenated into a vector to represent the point. Then, each video's vectors are grouped into four clusters which their centers are considered as the main directions and scales for an action. Moreover, dissimilarity of two videos is calculated by utilizing a novel fuzzy distance measure between their group centers. The experimental results on the most used UCF YouTube dataset with 11 action categories illustrated that, in contrast to the Bag-of-Words model, our method can perform better with less computational cost.
