I think they are both correct but have different meanings.
I cannot altogether understand the sentence. [ I don't understand the sentence fully or completely.]
I cannot understand the sentence as a whole. [Athought I understand parts of the sentence, when I consider the sentence all at once (as a whole) the meaning escapes me.]
You should use the one that has the meaning you desire. I agree with AngelEyes, that without context the second seems much more likely since you've already said that you understand the parts (words). But, the first is not impossible.