Men can expect to build 35 to 45 pounds of muscle over their lifetime, and 20 to 25 pounds for women. How easily we build muscle depends on a range of factors, personal trainer Mike Matthews said.