We've had some interesting neighbours over the years we've lived in our current house.
It's a new build semi built fairly recently, we met the couple who were moving into the adjoining house before we had even moved in and they seemed nice enough. Both worked as dentists locally and they were about to have their first child just like we were, so that felt like a nice coincidence, hopefully the kids would play and be friends we thought. Then soon after we moved in we started to hear arguments between the pair of them, nothing too major at first just a bit of shouting, but after not too long they started to get really loud and the arguments would be accompanied by loud bangs, stomping etc. So as this was getting worse and more frequent we started reporting them to the police, my wife used to work with victims of domestic abuse and by this point we were pretty certain that was what was going on in this case, however despite my wife's experience and knowledge on the subject it was difficult to get the police to take things seriously as we live in a decent area of town and these were two nice middle-class dentists - surely there couldn't be anything bad happening really?
Eventually after enough harassment the police properly got involved and yep - it turned out he was beating the crap out of her and generally being a complete dick - not to mention creating a pretty horrible environment for a baby to be brought up. In the end he got removed, she stayed and has managed to get her life back together thankfully, so happy ending - but not necessarily the scenario anyone dreams of living next door to!
The house on the other side of us, we share a fence but not a wall, used to be owned by what seemed like a nice couple in their early fifties. Both were friendly and active, we would see them going out to play tennis together regularly etc. After talking to them some more it turned out that he was an ex semi-pro tennis player/coach and she was one of his students and they had both left their previous partners after having an affair during their lessons together!
Eventually the tennis couple moved out and they sold the house to a group of girls who had just left university (I assume one of them had rich parents and bought the house for them to share), sadly the only window we had visibility of from our house was onto their stairs and in the three years they had been there I can report no noteworthy displays of exhibitionism!