With exponentially increasing the number of digital documents, text classification has become a major task in data science applications. Selecting discriminative features highly relevant to class labels while having low levels of redundancy is essential to improve the performance of text classification methods. In this paper, we propose a novel multi-objective algorithm for text feature selection, called Multi-Objective Relative Discriminative Criterion (MORDC), which balances minimal redundant features against those maximally relevant to the target class. The proposed method employs a multi-objective evolutionary framework to search through the solution space. The first objective function measures the relevance of the text features to the target class, whereas the second one evaluates the correlation between the features. None of these objectives use learning to evaluate the goodness of the selected features; thus, the proposed method can be classified as a multivariate filter method. In order to assess the effectiveness of the proposed method, several experiments are performed on three real-world datasets. Comparisons with state-of-the-art feature selection methods show that in most cases MORDC results in better classification performance.