A new wavelet‐Hilbert transform based sparse component analysis (WHT‐SCA) method is presented for online system identification in indeterminate conditions. The instantaneous phase ratios of output signals are obtained by using a wavelet‐Hilbert transform based filter; and the out‐of‐phase data, that causes errors in identification accuracy, is detected and eliminated. Then, modal parameters of the structure are identified through existing relationships between the dispersion of filtered data in the frequency domain. Subsequently, to demonstrate the capability of the online identification, a new controller is introduced by combining the WHT‐SCA and a semi‐active tuned mass damper (STMD), resulting in creation of smart structures. The performance of the proposed method and controller is investigated through examples. The results demonstrate that, modal parameters of structures are identified accurately even with noise contamination and limited number of sensors. Also, the STMD is effectively robust against any variations in modal parameters of the structure.