A goal on the stroke of half time from former Hereford player George Lloyd has seen Cheltenham lead 2-1.
Lloyd lashed the ball home after Brandon Hall saved from substitute Chris Clements.
It looked like Hereford were going to go into the break level pegging after Reece Styche had equalised.
Styche was on hand to stroke the ball past Scott Flinders in the 41st minute after Rowan Liburd set him up following good work from Jordan Nicholson.
Hereford were good value for the goal after coming back into the game after Will Boyle opened the scoring for Cheltneham from a corner in the 19th minute.
Their best chance fell in the 33rd minute when good work from Simeon Maye set up Liburd, but after rounding the ball past Flinders, Grant Horton was on hand to to clear it off the line.