You are correct, Qittat Ulthar, that it does not necessarily mean the bottom shelf, but I can't seem to think of a way to express it, so Faylasoof's suggestion may have to do. Or perhaps you could say something like الرف الذي تحت الرف الأول/الثاني/الأعلى مباشرةً.
I'm not sure تحته is used the way you used it. Rather the word أدناه is used to mean below in that sense. I also wanted to mention that phrases such as أدناه and أعلاه are, according to my knowledge, only used in formal written contexts in phrases such as المذكور علاه (previously mentioned) and الموقع أدناه (the undersigned). I don't think they are used to describe the position of physical objects in relation to each other. However, I might be wrong.
Well, if you have a bookcase with six shelves, for example, and you have a row of books all by one author on the first shelf, but this row continues on the second shelf, you could say it is on the "top shelf and the shelf below". So "the shelf below" there means the shelf directly under the shelf you mentioned before.