I was told that both "21st century" and "21th century" were common expressions, but I don't think the latter is grammatically correct. Is it okay to use that in written or oral English?
You were misinformed. 21th is incorrect. The -st of 1st, 21st, etc represents the end of the word first.
"21th" shows up in some published books, but it's very rare (ngram). As JulianStuart points out, it stands for a word that ends in "first", so using "th" instead of "st" doesn't make sense.
I would say that 21th is an error. I have done this myself when I edit dates, say from '25th' and then forget to change the ending. The digit 1 in its ordinal form is always first, except the mathematical expressions lie (n + 1)th.
