的 is usually left out here.If you said "我们在谁的家见（面）", would it sound natural too? (or is the 的 usually left out here?)
I think 家 is kind of a special concept, different to 笔 or 钱. Your can always switch or lose your 笔 or 钱, but basically you can't switch or lose your 家.的 is usually left out here.
@zhg: I just read your post #5 again. I don't know why 的 is usually left out here, but I don't think it's because of the syllable.
We do say 这是谁的书，我的笔找不到了，你的钱掉了...and there are many other examples...