I was told that both "21st century" and "21th century" were common expressions, but I don't think the latter is grammatically correct. Is it okay to use that in written or oral English?
"21th" shows up in some published books, but it's very rare (ngram). As JulianStuart points out, it stands for a word that ends in "first", so using "th" instead of "st" doesn't make sense.
I would say that 21th is an error. I have done this myself when I edit dates, say from '25th' and then forget to change the ending. The digit 1 in its ordinal form is always first, except the mathematical expressions lie (n + 1)th.