Actually, it was more like... he left, came back with no memories from the past, left again, came back from the same time period as the second time just without memories, left again, came back with memories from his second stay, left another time but came back with those same memories... and then disappeared again, came back from the future without memories, and has stuck around ever since.

[DEEP BREATHS. God, that was long winded, but then, that's just how it's been with Sora. Confusing, too.]

Point is, this place is unpredictable when it comes to that kind of thing. You never know when someone'll return or, if they do, whether or not they'll actually remember the place.