This is largely speculative, but I'll give it a shot.
Eventually most Slavic languages settled on a triangular five-vowel system consisting of
i,
e,
a, o and
u, with some preserving a sixth vowel,
y, which sits neatly in the triangle between
i and
u, or
ə, which also sits neatly between
e and
o.
Yat doesn't quite fit into the triangle. It would have to come either between
i and
e (As in Czech, Slovak, Sorbian and Western South Slavic, that would be /e/) or between
e and
a (As in Polish and Eastern South Slavic, that would be /æ/). This is based on the vowels it merged with, and I ashamedly admit that I have no idea what it's pronunciation would have been in East Slavic. Maybe also between
i and
e, considering it's fate in Ukranian?
So, there's three solutions on how to accommodate yat into the vowel triangle: 1. Turn it into a vowel rectangle, with closed
i and
u, mid
e and
o, and open
æ and
a. This is the default solution for modern Kajkavian dialects, which actually preserve yat as a separate vowel, closed /e/. 2. Turn it into a bigger triangle by innovating a distinction of open /ɔ / and closed /o/ to correspond to open /ɛ/ and closed /e/, so that there's closed
i and u
, closed-mid
e and
o, open-mid
ɛ and
ɔ and an open
a. This is the solution for Slovene and some modern Kajkavian dialects, and even those which today have a six-vowel rectangle probably used to have a seven-vowel triangle. 3. Get rid of yat. This happened in most other languages.
It's also not quite true that yat is the only vowel which has many reflexes. In Czech, the front nasal vowel *ę has even more reflexes: sometimes it becomes
a, sometimes it becomes... ironically, yat! Which then becomes
i or
e. So 5 is
pět, but 5th is
pátý. Similarly, in Čakavian it becomes
e or
a. And in Slovak the yers become either
e,
a or
o without anyone being completely sure under what conditions.
As for what yat's IPA value would be, that's even more speculative, since it's outcomes in different languages suggest two possibilities: either it was a closed /e/ between
i and
e , either it was an open /æ/ between
e and
a. It might have also been a diphthong, since it often becomes one in daughter languages. Which one of these is older? It's difficult to say, and I've read on this board ages ago that it might have been both for a time, depending on it's origin... Apparently there's some loanwords in Finnish which suggest /æ/ (
määra from *měra), and considering that in
some dialects of American English /æ/ has become /eə/ while skipping /ɛ/, the same may have happened in Slavic. So, but remember this is just speculation, I'm leaning for /æː/, which then becomes a diphtong /æa/ which then in some areas becomes more closed /eə/, which then becomes the varied reflexes in Western South Slavic and Czech.