This paper develops a novel approach to model the Interline Power Flow Controller (IPFC) with the purpose of enhancing the power system dynamic stability. The dynamic behavior of the IPFC is modeled using a new and detailed current injection model. On the basis of designing a supplementary damping controller, the effectiveness of the proposed model in robust damping of the oscillations is evaluated. Thus, the problem of attaining the damping controller parameters transmitted into an optimization process which is solved using Particle Swarm Optimization algorithm (PSO). The PSO has a strong and reliable capability to find out the optimistic solution. The optimization procedure is performed in a multi-machine power system and under various operating conditions. Assessment the derived results from the nonlinear time domain simulation and through some performance indices with considering to a severe transient disturbance clearly indicates the major performance of the proposed model and the model based designed controller in improvement the system stability margins. Moreover, to identify the most suitable IPFC control signal, a precise evaluation of the employed indices is accomplished. Numerical results verify the superior stabilization effect of the m1 (one of the IPFC control signals) in the wide range of operating conditions.