The Olive branch was a symbol of peace or victory. It was used mostly by the Federation when they sent envoys on missions of peace to various planets and governments.
The Federation seal features two olive branches. (DS9 novel: The Heart of the Warrior)
History
The Olive branch was used mostly as a initiative for peace by Starfleet in the 23rd century
In the 2261 of the Alternate reality, Capt. James T. Kirk decided to extend a olive branch towards the klingons when sent a distress call from Khitomer when the Romulans attacked. (TOS - The Khitomer Conflict comic: "Part 1")
In 2293 of the Prime reality, Capt. Kirk and his crew of the USS Enterprise-A were then chosen by the C-in-C to act as the Olive Branch to support Klingon Chancellor Gorkon's peace initiative.(TOS movie, novelization & comic adaptation: Star Trek VI: The Undiscovered Country)