Tengo que contestar en ingles. In this sentence "took off the gloves" figuratively means taking off boxing gloves. In means that the person is getting more angry about something and that the response is getting more serious.
stevea is correct. Si pegas a alguien con guantes de boxeo, no hace tanto daño y por eso se usan en deporte donde el objetivo es simplemente ganar. Quitarles significa un verdadero combate con la intención de hacer daño.
Not just boxing but in chivalry as well, the ritual of a gentleman who felt slighted or insulted. He would remove a glove, slap his opponent in the face and throw it to the ground. This indicated he was very angry and willing to fight for his honor.