It's actually not the number of syllables that is being referred to, but the number of feet, and a foot is made up of two syllables. In the case of iambic tetrameter, the line is made up of four feet, with each on consisting of an unstressed syllable, followed by a stressed one.
Thanks for the input though!