Indeed. Truax (AIAA paper a few years back; title was something like
"The future of earth-to-orbit propulsion") argued that complexity was
the primary cost driver and size itself was a relatively weak factor
(but of course there are the points at which size becomes a complexity
driver: as when your handling infrastructure becomes more than "pick
it up and walk to the pad with it"...)
