In AE, we would say either "you're letting the cold out" or "you're letting the heat/hot air in." Colloquially, they mean the same thing, although I have a feeling there's some law of thermodynamics that would that say one or the other (probably "letting the cold out") is scientifically inaccurate.
Interestingly, in the winter one hears "you're letting the cold in" much more often than "you're letting the heat out" ... although fathers have been known to say to their children, "shut the door! Are you trying to heat the whole neighborhood?!" I suppose some father somewhere has expressed the same sort of sentiment in the summer as "shut the door! Are you trying to air-condition the whole neighborhood?!"