From Dictionary.com: boil down,
a.to reduce the quantity of by boiling off liquid.
b.to shorten; abridge.
c.to be simplifiable or summarizable as; lead to the conclusion that; point: It all boils down to a clear case of murder.
an instance of intense argument (as in bargaining)
I could be very wrong, but I think your colleague is trying to say that he didn't understand your answer completely, and it looks to him like you're arguing/bargaining/fighting over something - what word fits there, you should know from the context.