In the recent decade, the robustness, reliability and operability of semi-active tuned mass dampers (STMD) against frequency changes in linear structures has primarily been investigated in the frequency domain. Under severe environmental excitations such as strong earthquakes or winds, structures are expected to respond in the nonlinear range due to material nonlinearity or occurrence of damage which necessitates time domain analyses. In this study, a new proposed STMD controller is presented based on integrating damage detection and semi-active control strategies. The variable stiffness of the STMD is provided by a semi-active independently variable stiffness (SAIVS) device. The stiffness variation of the SAIVS device is implemented by adapting the frequency of the STMD to the fundamental frequency of the bare structure (without STMD), which is obtained using real-time tracking of the damage in the structure based on the measured dynamic responses of the coupled main structure and the STMD system. In this paper, the structural parameters of the bare structure are estimated using the damage detection algorithm based on identified system Markov parameters (DDA/ISMP) method. The effectiveness of the proposed scheme is evaluated via numerical analysis of a nonlinear ten-story building considering seven performance criteria under real earthquakes. In comparison to the passive case, the results reveal that the proposed STMD controller is robust against any structural frequency variations which effectively mitigates damages and nonlinear dynamic responses of the structure under strong ground motions.