He was born at the rectory, Kilsallaghan, County Dublin, of which parish his father, John Gregg, was then rector, on 3 May 1834. His mother was Elizabeth Law. He was educated at Trinity College Dublin, where he graduated as junior moderator in mathematics in 1856, being awarded M.A. in 1860. In the same year Gregg was ordained for the curacy of Rathcooney, County Cork, and three years later was appointed rector of Christ Church, Belfast, an important cure which brought him in touch with the working-class population of the north of Ireland. In 1862 he returned to the Diocese of Cork as rector of Frankfield and chaplain to his father, then Bishop of Cork. Frankfield was perhaps the place with which he was to have the closest connection: his wife Elinor was a native of Frankfield, and they are both buried there. In 1865 he became rector of Carrigrohane and precentor of Saint Fin Barre's Cathedral, Cork. Here he quickly acquired a high reputation for administrative ability, as well as for the qualities of sound judgment, moderation, and good sense by which he was subsequently distinguished in the episcopal office. In the controversies which followed, the disestablishment of the Irish church, particularly in regard to the revision of the prayer-book, Gregg took the conservative side, but his influence was uniformly exerted in a conciliatory spirit. Gregg's principal service to his church at this time lay in devising for his own diocese of Cork the singularly successful financial plan which became the foundation of the financial system of the disendowed Church of Ireland, and on this and other occasions he showed a remarkable talent for finance. In 1873 he was presented by the University of Dublin with the degrees of B.D. and D.D., in recognition of his services to the Church of Ireland. In 1874, Gregg was appointed Dean of Cork, and in the following year was selected by the Irish bishops to succeed James Thomas O'Brien in the diocese of Bishop of Ossory. Gregg, at forty-one years of age, thus became a member of the episcopal bench while his father was still Bishop of Cork, Cloyne and Ross. On his father's death on 26 May 1878, the synods of Cork, Cloyne and Ross at once selected Gregg to succeed him. As Bishop of Cork, Gregg's most noticeable work lay in the completion of the beautiful cathedral of St. Finn Barre, which had been rebuilt during his father's episcopate at a cost of over £100,000 ; but he also won a deserved reputation not only for administrative efficiency, but for a statesmanlike grasp of church problems which opened the way to the highest office in the Irish church. On the death in 1893 of Primate Robert Bent Knox, Gregg was selected to succeed him as Archbishop of Armagh and Primate of all Ireland. He married in 1863 Elinor Bainbridge of Frankfield, and had two children, John and Amy. Amy in 1898 married as his second wife Robert Walsh, Archdeacon of Dublin. He died suddenly at the Palace, Armagh, on 10 January 1896, after scarcely two years' enjoyment of the primacy, and was buried at Frankfield beside his wife Elinor. His nephew John Gregg became in his turn Archbishop of Armagh.