I'm going to have to agree with the NIC recommendation. We saw a 50% drop in overall package build time simply by replacing a 100Mb NIC (operating at 100/Full Duplex) with a fiber based GigE card. You need to look at cutting down on latency during the package build process. Updating statistics on all of the CO tables can help if that hasn't been done in some time. Also, placing the tablespaces on disk subsystems that can deliver optimal read performance is another thing to look at. Obviously NAS storage for Central Objects usually isn't the way to go for quick package build times.
Running a full package on a fat client as opposed to the deployment server can also slow things down somewhat. It just depends on how your server is configured (i.e. optimized for applications or file serving) and whether or not your fat client has the same speed connection as the server hosting the Central Objects datasource and your deployment server. Best bet is to have the database and deployment servers on the same subnet, same switch if possible and setup with the same speed network interfaces. It might even be a good idea to use the same make and model cards on a network switch known to work well with the NIC hardware. I've heard of issues with certain Broadcom chips not playing well with Cisco switches, but 3COM works great. Same goes for other vendors and 3COM switches, so it isn't a Cisco problem. I read an article the other day about the dirty secrets behind GigE driver implementations, so be warned that "your mileage may vary".
Try defragmenting your deployment server drives where you've installed the application. As others have said, turning off AV software and realtime scanning agents can speed things up tremendously. Making sure your deployment server drives are also up to snuff as far as high performance read/write operations is also a good idea. A fast caching RAID adapter can help somewhat.