"In the grand scheme of things" is an idiom. It means that when you put things into perspective, taking EVERYTHING else into account, sometimes what has previously been considered significant isn't quite, well, significant.
It's a variation from "in the scheme of things", meaning in light of how things – all things – generally work. In the religious sense, the scheme is seen as God's scheme, his great plan for everything. Otherwise it's just an overview of the overall situation. In other words, it's the big picture.
In the scheme of things, for example, humans are insignificant. Here, humans are being discussed with respect to all species and in terms of the cosmos as a whole. After all, we are just a small part of what's going on with Mother Nature on Earth. Indeed, the Earth itself is tiny in the scheme of things, in that it's a small piece of rock hurtling in the vast solar system, which itself is in fact a tiny system in the galaxy among innumerous other galaxies in the whole universe...
There are other variations from "in the scheme of things" including "in the greater scheme of things", "in the overall scheme of things" and of course "in the grand scheme of things". Here are media examples to give you a chance to see each one of these variations in action.