The pacy former Wolverhampton Wanderers player is well-known for his bulging muscular frame, and teammate Crysencio Summerville last week posted a video on social media showing Traore bench-pressing 145 kg.
"It's incredible (his physique), it's genetics," Espirito Santo told a news conference on Tuesday ahead of his relegation-threatened side's crucial clash at London rivals Fulham.
"(But) his genetics has been like this for some time now and he should avoid the gym. I've told him to stay out of the gym.
"It's one of the things that I think he needs to realise. It's enough weight that he carries."
Spaniard Traore, 30, joined West Ham from Fulham in January and has made four substitute league appearances for them. Espirito Santo, who previously worked with him at Wolves, believes Traore could play a key role in the relegation battle.
"Traore's unique. There are not many players in world football with his ability, pace and skills in one-versus-one situations," he said. "He's a talent that we have to take advantage of, but it will take time.
"He has to adapt and understand the dynamic of the team. But he's proven in England and in the Spain squad not long ago, so we're talking about a high-level player."
Traore, who came through Barcelona's ranks, has been recorded reaching speeds of 37kph and once claimed he would beat Olympic great Usain Bolt over a short sprint on grass.