In “elegy written in a country churchyard” by thomas gray, the speaker says that the forefathers never became rich, famous, or powerful. what reasons does he give?

He gives a reason for this that from innocence the forefathers "grew" their safety  rather than by "power and genius" perhaps meaning that having a good heart and meaning well for other people results in more "safety" in life than "power and genius" which brings to mind the demise of Conrad Black who sought Power and Genius I would say (perhaps as an addiction to power) that it led him to break the law and end up in jail.