This is how my agent explained it at some places I looked at before I bought my first house... assuming you have a decent neighbour it shouldn't be an issue. However if your neighbour is a prick then it can get ugly.
There was a house by mine that has a mutual driveway. The one guy blocked the drive and wouldn't move, so the other guy got a front endloader from the gravel pitt and moved it for him. Cartwheeled it right into a field.