On some maps it's possible to built a lot of them, so there should be some kind of restriction. I can remember that there was already an idea to limit the number of castles to the TC's you possess. Here's another idea:
The turns you need to built the great building will increase with 2 turns after each building. So for example, the mansion will first cost 13 turns, second mansion will cost 15 turns. Third mansion cost 17 turns. Than 19, 21, 23 etc
Maybe 2 turns is too much, than it should be +1 turn after each building.
