Marcus Dinanga scored the only goal of the game at Edgar Street in the 77th minute.
But a late push from the Bulls, including keeper Curtis Pond going close with a header, couldn't see them net an equaliser and force the game to penalties.
The result means National League Gateshead have booked themselves a place in the quarter finals.