The most important problems with exploiting Artificial Neural Networks (ANNs) are to design the network topology, which usually requires an excessive amount of expert's effort, and to train it. In this paper, a new evolutionary-based algorithm is developed to simultaneously evolve the topology and the connection weights of ANNs by means of a new combination of Grammatical Evolution (GE) and Genetic Algorithm (GA). GE is adopted to design the network topology while GA is incorporated for better weight adaptation. The proposed algorithm needs to invest a minimal expert's effort for customization and is capable of generating any feedforward ANN with one hidden layer. Moreover, due to the fact that the generalization ability of an ANN may decrease because of overfitting problems, the algorithm utilizes a novel adaptive penalty approach to simplify ANNs generated through the evolution process. As a result, it produces much simpler ANNs that have better generalization ability and are easy to implement. The proposed method is tested on some real world classification datasets, and the results are statistically compared against existing methods in the literature. The results indicate that our algorithm outperforms the other methods and provides the best overall performance in terms of the classification accuracy and the number of hidden neurons. The results also present the contribution of the proposed penalty approach in the simplicity and generalization ability of the generated networks.
