Hello, Pentapoli; if you are looking AT the sky, you are looking at the sky itself. If you use "in the sky", you are talking about something that is in the sky. "Look at the clouds in the sky" or "Look, up in the sky, it's a bird...it's a plane...no, it's superman!" (Which is actually the way the old Superman program began!)
Or, "If I look up in the sky I don't see anything but blue, as far as the eye can see"--again, you would be looking for something, not the sky itself.
Hope that helps!