Florence Ballard was born in Detroit in 1943. The ninth of fifteen children, Ballard's parents Lurlee (née Wilson) and Jessie Lambert Ballardhad migrated from Rosetta, Mississippi, to participate in Detroit's then booming job market. Jessie Ballard eventually found work at General Motors but still struggled to take care of his growing brood. The Ballards moved constantly around Detroit, living in 7 Mile for a brief time and then living at the Black Bottom projects. Ballard's family eventually settled at the then-newly developed Brewster-Douglass housing projectsby the time Ballard was fifteen.

Ballard's cousin was rock and roll and soul music pioneer Hank Ballard. Ballard began singing in church at an early age. Ballard, often called "Flo" by family and friends, also acquired the nickname "Blondie" due to her light auburn hair and fair complexion that reflected her mixed heritage. In 1958, Ballard met Mary Wilson and became acquainted with her after they participated in the same talent competition.

Milton Jenkins, a local man then best known for his work with the all-male group the Primes (who became The Temptations), was scouting for girls to become members of his group's sister act, the Primettes. Jenkins took an interest in Ballard's voice after Ballard auditioned for him and Primes members Paul Williams and Eddie Kendricks. Jenkins recruited Ballard as the first member of the Primettes and asked her to enlist other members. Immediately Ballard convinced Mary Wilson to join the group; Wilson then contacted fellow neighbor Diana Ross. Eighteen-year-old Betty McGlown, who was dating Paul Williams at the time, rounded out the quartet. In 1960, McGlown left and was replaced by Barbara Martin. Martin eventually left in 1962, and the group decided to remain a trio.

Described by Wilson and friend Jesse Greer as having been a generally happy if somewhat mischievous and sassy teenager, Ballard experienced a change in personality, from which she seemingly never recovered, as the result of an incident that occurred in the summer of 1960. Leaving a sock hop at Detroit's Graystone Ballroom one evening, Ballard accidentally was separated from her brother Billy, with whom she had attended the event. Accepting a ride home from a young man she felt she recognized, local high-school basketball player Reginald Harding,^[3] Ballard was instead driven north to an empty parking lot off of Woodward Avenue. There, Harding raped Ballard at knife point.

After weeks of sequestered silence that confused Wilson and Ross, Ballard finally told her groupmates what had happened to her. The girls were sympathetic but as confused as Ballard herself, whom they had considered strong-willed and unflappable. Consequently, Ballard's assault was never mentioned again, either in clinical therapy or in social conversation^[4]—something that Wilson believes heavily contributed to the more self-destructive aspects of Ballard's adult personality, such as her cynicism, pessimism, and fear or mistrust of others.