What i mean by not comparable is if a scout kills an assassin you only lose 4 turns of 'production'... if an assassin kills your castle you are trading a 16 turn castle for 4 turn unit. You lose 12 turns of 'production'. Not comparable as it is much worse to lose 12 turns of 'production' than 4.
Sure castles can be built faster, eventually. But how many turns did it take to make all those builders? And it still takes up production, even if it is builder production.
Yeah - You'll loose 12 turns of production instead of 3. OK. But by sacrificing 4 turns of production instead of using 1 turn production unit (let's say sacrificing, because it's close to useless if there is no invisible units).
When it happens 4 times it's the same production output.
Please do not use logic of wealthy people advocating head tax as the most fair.
Currently the assassin does the trebuchets job, killing defensive structures, more efficiently some of the time. Trebs are 7 turn production from vulnerable, 16 builder turns to build Castle, and take multiple shots to take down a castle. On the other hand if you can get an invisible unit to the Castle walls you take it out with no danger. Of course, trebs dont get used up when shooting, but the point still stands, considering all the other targets assassins would have even after you took away their ability to one shot castles.
And yet, somehow trebuchets never stop being used. They are considered one of the best units. Why?
Might aswell call it the Townhall then, looking at your ideas of what it should do, even though Castles were much of the time solitary defensive structures, surrounded by farms and peasants.
From wikipedia:
Scholars debate the scope of the word castle, but usually consider it to be the private fortified residence of a lord or noble.
Lords or nobles rarely lived surrounded only by farms and peasants. Usually there were cities or towns formed around the castles.
Townhall on the other hand - has nothing to do with nobility. It also came much later.
Making it unusable shouldnt make it collapse, something like disabling production inside for 4 turns, or killing a unit inside makes sense.
Making it so unusable, that there is nothing, that makes a castle do what it does - it's the equivalent of collapse.
If all people inside are killed, production facilities, machines, defensive equipment destroyed or stolen, why wouldn't it be equivalent of total collapse?
When walls are attacked they are "evaporated", while in reality there would be whole walls with a single breach - just enough to travel through it.
The representation of castle is destroyed to make it clear, that You need to use the same amount of resources to rebuild it.