Huang was born in Chuansha, Shanghai, in the final years of the Qing Dynasty. He was a distant relative of Huang Yanpei. He was accepted into Tsinghua College in 1916 and was introduced to Western music there. After his graduation in 1924, Huang went on to study psychology in Oberlin College in Ohio, United States. In 1928, he was accepted into Yale University, where he studied Western music. In Yale, he composed the overture In Memoriam, which is the first large-scale orchestral work by a Chinese composer. In 1929, Huang returned to China and taught in the University of Shanghai, National Music College and other music schools. In 1935, he established the Shanghai Orchestra, the first all-Chinese orchestra. Some of his students, including He Luting, Ding Shande, Zhu Ying, Jiang Dingxian, Lin Sheng, Lin Shengxi and Liu Xue'an, became famous musicians later.