This paper introduces a knowledge-based semantic image segmentation which extracts the “object(s)-of-interest” from the image. Image templates are the high-level knowledge in the system. The major contribution of this work is the use of the “Global Precedence Effect” (forest before trees) of the human visual system (HVS) in image analysis and understanding. The “object-of-interest” is searched for hierarchically through an irregular pyramid by an affine invariant comparison between the different region combinations and the template starting from lowest to the highest resolutions. The global/large size objects are found at lower resolutions with significantly lower computational complexity.