Thank you!The OED has the fused version as the headword too, but I note that the hyphenated version is not uncommon in their recent quotations.
2005 R. Zadok Gem Squash Tokoloshe i. viii. 63 Startled at being caught out in my game of peek-a-boo.
1993 J. Meades Pompey (1994) 258 Down there, end of a stage-left aisle, twelve rows back, seats J12 and J11—a girl with a peek-a-boo flicks away a squaddy's hand from her thigh.
1998 R. Newman Manners 202 His right hand goes to his jacket side-pocket. Peek-a-boo. He drops a black leather wallet back in his pocket again.