"Towards the argument" or "for the sake of argument". That is, let's discuss this (construct an argument) on this assumption.
I don't actually know 'ad argumentum' as a technical term. Similar Latin phrases used in English (probably not used much any more) are of the type 'argumentum ad __', meaning "argument addressed to": an argumentum ad hominem is addressed to the person (their peculiarities), an argumentum ad crumenam is an argument addressed to the wallet (support this position or it'll cost you more), and so on.
So weird, I wound up here because of the same episode of Big Bang Theory. I work in the (U.S.) legal industry and have come across the term to assume, arguendo (i.e., for the sake of argument), which obviously carries a meaning that would fit exactly with the context in that moment in the episode. And I have likewise been unable to yield anything from a Google search of "ad argumentum" or "ad arguendum" (the latter of which is how the actor actually pronounces it, as I have the episode paused and have listened back). Therefore, I intuit that it is simply an error either in writing or on the part of the actor. Any further confirmation or invalidation is greatly welcomed, though.
Further to my comment above, I think if another person makes their way to this thread as a result of that episode we can probably safely say that it is not a true expression, but an error on the part of the writers or actor.