Honestly I consider removing the seat in the future buggy since I really don't like people riding with me anyways. I like wheeling with my group, but I hate having to "entertain" someone sitting in the pass seat. I'm not much for small talk and hate having to explain everything. Buggy time is my time.
Having said that, I wouldn't get rid of your pass seat altogether, I'd just swap it out if you ever needed it. I wouldn't swap it on a trip, but if you knew you had a run coming up and the woman or a friend was going with you, you could swap it in at home kinda thing.
You can also just ignore everything I say since my future buggy will probably be a 4 seater so I can take both my boys with me if they ever go.