In response to the fast propagation of videos on the Internet, Content-Based Video Retrieval (CBVR) was introduced to help users find their desired items. Since most videos concern humans, human action retrieval was introduced as a new topic in CBVR. Most human action retrieval methods represent an action by extracting and describing its local features as more reliable than global ones; however, these methods are complex and not very accurate. In this paper, a low complexity representation method that more accurately describes extracted local features is proposed. In this method, each video is represented independently from other videos. To this end, the motion information of each extracted feature is described by the directions and sizes of its movements. In this system, the correspondence between the directions and sizes of the movements is used to compare videos. Finally, videos that correspond best with the query video are delivered to the user. Experimental results illustrate that this method can outperform state-of-the-art methods.