Currently, thyroid-associated ophthalmopathy (TAO) lacks effective treatment due to our lack of clarity in its immunopathogenesis. Orbital fibroblasts play a key role in altering inflammation and immune response in TAO, and are considered as the key target and effector cells in its pathogenesis. The orbit infiltrating CD34+ fibrocytes add on to the process by expressing high levels of autoantigens and inflammatory cytokines, while also differentiating into myofibroblasts or adipocytes. This review focuses on the role of orbital fibroblasts and CD34+ fibrocytes in the pathogenesis of TAO, highlighting the basis of emerging treatments.