I actually had to do some searching myself to find out what 청량음료 refers to technically, and it my surprise it does include non-carbonated drinks, such as lemonade. So to answer your question, no it does not necessarily have to be carbonated. I guess if you take into consideration the sino-Korean etymology of 청량, 淸凉, which simply means "clear and cool", it is not all too unreasonable. There in fact is a specific term that refers to carbonated drinks: 탄산음료.
Then again it must be noted that many natives more often than not may interpret "청량음료" as "carbonated drinks" due to how overwhelming majority of what is referred to as 청량음료 is carbonated. However, technically if you wanted to specifically refer to a carbonated beverage product you'd say "탄산음료". Hope this helps.
Just wanted to add examples, which might help.
For me (a native Korean), typical 청량음료 means coke, sprite, 7-up, mountain dew, fanta, and all those unhealthy sugary carbonated cold drinks. Taking a a bit tricky example, I am not quite sure if Snapple is 청량음료. It seems cold (often times), unhealthy and sugary, but the problem is it's not carbonated. I ask myself, and I am still not sure if it's 청량음료. Maybe, maybe not. The funny thing is that most of Koreans won't even care much as the word 청량음료 itself doesn't really have exact definition in their mind, either.