Most early recorders employed steel tape to record in, which made them heavy and difficult to use, or paper tape, which was easier to use but often broke.
Atlas' engineers doubled down on this idea, and outfitted their computer with 4 paper tape readers, 4 paper tape punches, and up to 8 magnetic tape drives.
Within about 50 years, software grew in complexity from machine code punched by hand onto paper tape, to object oriented programming languages, compiled in integrated development environments.
And they then wrote code, usually machine code, sometimes binary code, which was then sent by mail to a data center to be punched onto paper tape or card and then re-punched, in order to verify it.