"I failed the entrance exam" is the common way to say that you did not pass the entrance exam. It's probably the right way to say what you want to say.
English is so malleable, though, that almost nothing is wrong, per se. You could say, for instance, "I failed in the entrance exam... when I totally flubbed the essay question / to conjugate Latin verbs correctly." In that case "in the entrance exam" means "in the context of the entrance exam, while I was taking the entrance exam."
BE: I'd say " he failed the entrance exam". This means they didn't get in. Although I can understand why 'in' might be used, I don't think it is really correct. It certainly isn't necessary. I might use fail in an exam if the exam was one of several which make a whole. "He didn't get the certificate because he failed in the reading comprehension test". Even there I am not happy about it.
I might say that I failed in an exam if I got a B when I was expecting an A.
In an entrance exam I might say it if my grade wasn't good enough for me to secure my preferred choice of subjects.
There's also the possibility that I might pass various other criteria but fail in the exam (cf Hermione).
Normally I would say "fail the entrance exam".