Leslie James Harman began piano lessons at age four, and also sang in his local church choir. Harmonicas owned by his father were stored in the piano bench, and Harman tried playing them after his piano lessons ended. In time, he became capable in several other musical instruments, including guitar, electric organ, and drums.

Harman performed as a blues harmonica player and singer in Chicago, New York, and elsewhere before moving to southern California in the 1970s. There, his Icehouse Blues Band played alongside Big Joe Turner, John Lee Hooker, Freddie King, Muddy Waters, Albert King, B. B. King, T-Bone Walker, Lowell Fulsom, Eddie "Cleanhead" Vinson, Johnny "Guitar" Watson, and Albert Collins.

In 1977, he formed the James Harman Band. Over the years, their line-up has included Phil Alvin and Bill Bateman, who left in 1978 to form the Blasters; Gene Taylor, who departed in 1981, also to join the Blasters before moving on to The Fabulous Thunderbirds; and Kid Ramos. Alumni also included the late Hollywood Fats who, after leaving his own band in 1980, played alongside Harman for five years.

His debut solo album Those Dangerous Gentlemen was released on the Rhino label in 1987, with a succession of albums released on labels like Black Top, Hep Cat, and Cannonball.

His 2015 full-length Bonetime (Electro-Fi Records) was his first studio album in over twelve years. By the end of the year, he'd earned five 2016 Blues Music Award nominations: Best Album (Bonetime), Instrumentalist (Harmonica), Song (“Bad Feet/Bad Hair”), Traditional Blues Album (Bonetime), and Traditional Blues Male Artist.

In early 2021, Harman was receiving chemotherapy treatment for stage 4 cancer.