The pair looked in great spirits as they walked hand in hand and continued to enjoy married bliss