Born in Paris to an immigrant Polish Jewish father, Alter Mojze Goldman and a German Jewish mother, Ruth Ambrunn, Goldman was the third of four children. As a child, he began his music studies on the violin, then the piano. In 1968, he abandoned his classical music studies for "American Rock & Roll" as well as folk music, listening to The Beatles, Jimi Hendrix or Aretha Franklin, and emphasizing the guitar. He also earned a business degree from the École Des Hautes Études Commerciales, commonly known as EDHEC, in Lille. In 1972, he met Catherine, his first wife, with whom he had three children. He first entered the French music scene as member of a progressive rock group named Taï Phong ("great wind, typhoon" in Vietnamese), which released its first album in 1975. Their first song to be a moderate hit was "Sister Jane." After singing four years and three albums in English with Taï Phong, Goldman was determined to do it alone and write and sing in French.
In 1981, Marc Lumbroso heard his recording "Il suffira d'un signe" on the album Démodé and signed him to a five-album contract with Epic Records. In 1982, his first hit album Minoritaire, which included the hit song "Quand la musique est bonne", was released; subsequent albums have all been successful. In 1987, he recorded the hit single "Là-bas" with Sirima, which remained a popular song throughout the years. From 1990 to 1995, he recorded 2 studio albums, one live album and released several singles (such as "Nuit", "À nos actes manqués", "Né en 17 à Leidenstadt" and "Tu manques") with Carole Fredericks and Michael Jones, which were successful. Several of these songs were recorded with English versions, but did not find much success in England or the United States. In 2007, he recorded the hit "4 Mots sur un piano" with Patrick Fiori and Christine Ricol.
Jones is a Welsh singer who had settled in his mother's native France. He and Goldman scored a hit in the early 1980s with "Je te donne" ("I give you"), a bilingual song with Jones singing the English lyrics and Goldman the French. Fredericks was an American singer, the sister of Taj Mahal who settled in France in the 1970s.
Jean-Jacques Goldman wrote the theme for "Restos du Coeur" the eponymous song performed by les Enfoirés as the theme of the charity created by Coluche, Les restos du coeur. Throughout his career, he has frequently written for other singers, notably Céline Dion : he wrote her whole albums D'eux (released in the US as The French Album) that is the best selling French album in history with 9.000.000 copies worldwide, S'il suffisait d'aimer and 1 fille & 4 types, and also composed the songs If That's What It Takes, I Don't Know and Fly of her album Falling into You, and Let's Talk About Love of the album of the same name. He has also collaborated with Johnny Hallyday, Patricia Kaas, Garou, Marc Lavoine, Gérald De Palmas, Patrick Fiori, Khaled ("Aïcha"), Lorie and Florent Pagny as well as famous jazzman Chet Baker, Supertramp's saxophonist John Helliwell, Englishman Joe Cocker, American king of soul Ray Charles and American songwriter Diane Warren.
On 19 November 2012, Génération Goldman, a tribute album to Goldman was released on My Major Company France and M 6 Music label with a number of artists interpreting Goldman songs.