A host of a Trill symbiont gains access to the memories of the symbiont; therefore, also those of all of the previous hosts.
When a symbiont was not inside a host body, they made use of electrochemical signals to communicate with others through the liquid in which they lived. A joined symbiont was also capable of communicating with its non-joined counterparts, by being submerged in the fluid breeding pools on the Trill homeworld. This provided a direct form of communication between the two, but not on the same level of sharing knowledge as that which a Joining was capable of providing.
While typically symbionts only communicated with and took Trills as hosts, it was possible for them to communicate with Humans, at least through extended contact. This was done through matching the symbiont's bioelectric patterns with those of the other individual. However, this required direct tissue contact between the nerves of the person with those of the symbiont. (DS9 novel: Time's Enemy)
Sometimes, symbionts were born with a rare biochemistry that made them unfit for joining. This occurred only once every hundred years or so. The last known symbiont with this affliction was Marh, who attempted to join with Trill Guardian, Aylim, in the 14th century. (ST - New Worlds, New Civilizations short story: "Aversion")
As symbionts grew older, they eventually became unable to take hosts and would sink to the bottom of the pools, where they would become caretakers for the Annuated. Eventually, these symbionts would grow to become Annuated themselves over a period of 20,000 years or more (DS9 novel: Trill: Unjoined).
In the 24th century humans could be joined with symbionts on a temporary basis, but was quite detrimental to the human hosts. (TNG episode: "The Host") By the 32nd century humans and symbionts could be joined together on a permanent basis with few detrimental effects to human hosts. (DSC episode: "People of Earth"). However, in some cases human hosts had trouble accessing the past memories of the symbiont following joining. Additionally not all Trill were accepting of non-Trill being joined to symbionts. (DSC episode: "Forget Me Not")
In the mirror universe, the Trill Symbionts were infected with a hostile parasitic intelligence. Emperor Spock ordered all the symbionts, save for Dax, exterminated, in order to stop them becoming a threat to the Empire. This has left Curzon Dax as the only joined Trill. (TOS - Mirror Universe novel: The Sorrows of Empire)
See Also: Trill symbionts for a list of symbionts.