Wilen was born in Nice, France; his father was an American dentist turned inventor, and his mother was French. He began performing in clubs in Nice after being encouraged by Blaise Cendrars who was a friend of his mother. His career was boosted in 1957, when he worked with Miles Davis on the soundtrack for the Louis Malle film Ascenseur pour l'Échafaud (Elevator to the Gallows). In 1959, Wilen wrote his soundtrack Un Témoin Dans la Ville and studio album Jazz sur scène with Kenny Clarke. He wrote a soundtrack for Roger Vadim's film Les Liaisons Dangereuses two years later, working with Thelonious Monk.[3