Genetic engineering is the practice of manipulating the DNA of a living organism, usually through the use of cloning.
Some of the earliest genetic engineering performed on Earth was by Dr. Sarina Kaur of the Chrysalis Project, who created augmented humans. These individuals—including Khan Noonien Singh—brought about the Eugenics Wars of the late 20th century, and as a result, human genetic engineering was severely limited by subsequent law. (TOS - The Eugenics Wars novels: The Rise and Fall of Khan Noonien Singh, Volume 1, The Rise and Fall of Khan Noonien Singh, Volume 2, To Reign in Hell: The Exile of Khan Noonien Singh)
Human genetic engineering remained controversial into the mid 22nd century, when Dr. Arik Soong stole a number of frozen Augment embryos preserved following the Eugenics Wars. He brought them to term and raised them through early childhood, until he was arrested and imprisoned. To his dismay, however, his "children" displayed many of the same antisocial tendencies as their predecessors. (ENT episodes: "Borderland", "Cold Station 12", "The Augments")
Several early Earth colonies continued to practice genetic engineering, even centuries after the formation of the Federation. (TOS - The Eugenics Wars novels: The Rise and Fall of Khan Noonien Singh, Volume 1, The Rise and Fall of Khan Noonien Singh, Volume 2; TNG episode: "The Masterpiece Society"; SCE eBook: Out of the Cocoon)
Following the formation of the United Federation of Planets President Thomas Vanderbilt convinced the Federation Council to adopt a general ban on genetic engineering that United Earth had in place throughout the Federation. This was done to appease the Klingons after their own genetic engineering program using human DNA failed and resulted in a great many Klingons being turned into QuchHa' Klingons in the process. The Federation did allow for treatment of birth defects through genetic engineering. (ENT - Rise of the Federation novel: A Choice of Futures)
Some genetic manipulation was still allowed to treat serious birth defects, but large-scale "eugenic" procedures were outlawed. Still, as late as the mid 24th century, illegal procedures were still being performed. When it was discovered that Julian Bashir had his genetic abilities enhanced by doctors at Adigeon Prime at age five, he faced possible dismissal from Starfleet, though ultimately his father, Richard Bashir, agreed to a prison term. (DS9 episode: "Doctor Bashir, I Presume")
The individuals that Richard and Amsha Bashir selected to perform genetic modification on Julian Bashir had done them correctly, leaving Bashir able to function in Federation society. Other people had gone to far less scrupulous individuals to have their children genetically modified. As a result, their children developed side effects to the treatment that left them unable to function in normal society. Many such children were institutionalized as a result. (DS9 episode: "Statistical Probabilities")
The Klingons attempted to build on the work on human augmentation to improve their own race. The results were deadly and ultimately led to a widespread alteration of the Klingons' physical appearance. (ENT episodes: "Affliction", "Divergence")