As Loob said it depends on situations and contexts in which you want to use these phrase.
At one situation these two are interchangeable - when you are talking about making something dirty or untidy - in other situation it's not. As in
i) Try to place the waste in proper place, don't make the room messy.
ii) Try to place the waste in proper place, don't make a mess of it. ( But saying that it's interchangeable I have to admit there are some slight difference in these two, this particular sentence seems more of making the 'waste' scatter or something like that. But if it scatters it definitely make the room dirty. But it depends on the context.)
But in other situations they are not interchangeable. Like
i) He tried hard but finally made a mess of his exam.
"make a mess" can be used in various situations and contexts.