Zierlein embarked on his coaching career as a graduate assistant and linebackers coach at his alma mater in 1970. After two years coaching at Fort Hays State, he spent the following six seasons as a high school assistant in Texas. In 1978, he returned to the college game at the University of Houston where he served as offensive line coach for nine seasons. In 1987, he got his first professional experience as an assistant with the Washington Commandos of the Arena Football League. When the Commandos went on hiatus the next year, Zierlein took a job at Tulane where he worked the next three seasons before returning to the pro ranks with the New York/New Jersey Knights of the National Football League's development league, the World League of American Football. With the Knights he held the title of offensive coordinator under Mouse Davis as well as coaching the offensive line. When the Knights folded in 1992, Zierlein once more headed back to college, this time at LSU. Zierlein spent two years at LSU, coaching the offensive line under head coachCurley Hallman. In 1995, he returned to Tulane, this time for what would be a two-year stint under Tommy Bowden. In 1997, he moved on to the University of Cincinnati as the running game coordinator and offensive line coach under Rick Minter. In 2001, Zierlein gained his first NFL experience with the Cleveland Browns. He served for four seasons as the Browns' offensive line coach under the Butch Davis regime. Zierlein's current Pittsburgh Steelers colleague Bruce Arians served as the offensive coordinator for the first three of Zierlein's seasons at Cleveland. He returned from a one-year absence from football in 2006 as the assistant offensive line coach for the Buffalo Bills under Dick Jauron. When Mike Tomlin was hired as the Steelers new head coach in 2007, he brought Zierlein to Pittsburgh to coach the offensive line. Tomlin and Zierlein had previously worked together in 1999–2000 with the Cincinnati Bearcats. Steelers assistant special teams coachAmos Jones was also a member of that Bearcats coaching staff. Zierlein was relieved of his duties with the Steelers on January 6, 2010.
E-mail gaffe
Zierlein gained unwanted notoriety in 2007 when he mistakenly forwarded an e-mail to a large number of high-level NFL employees, including commissionerRoger Goodell. The e-mail, which Zierlein had received from Steelers pro personnel coordinator Doug Whaley, contained an inappropriate video. Zierlein apologized for the gaffe, but was not disciplined by the team or the league.