I would translate אל as towards just for the sake of clarity, which necessarily implies movement and closing the distance.
So with verbs of movement of one physical object towards another it's (i guess) always אל
With pronominal suffixes אליי, אליה, אליכם etc. it's used both literally and figuratively, that is when reference to a person implies the person's place.
The difference between the connotation may transpire from context where אל in physical sense seems to be more pertinent to continuous actions while in figurative sense - to completed (perfective) actions, e.g.
מספר שניות הלכתי אליו - for a few seconds i was walking towards him
אחרי עבודה הלכתי אליו - after work i went to his place