Due to the spatial heterogeneity of land surfaces, downscaling is an important issue in the development of carbon cycle models when evaluating the role of ecosystems in the global carbon cycle. In this study, a downscaling algorithm was developed to model gross primary productivity (GPP) at 500 m in a time series over rugged terrain, which considered the effects of spatial heterogeneity on carbon flux simulations. This work was carried out for a mountainous area with an altitude ranging from 2606 to 4744 m over the Gongga Mountain (Sichuan Province, China). In addition, the Moderate Resolution Imaging Spectroradiometer (MODIS) GPP product at 1 km served as the primary dataset for the downscaling algorithm, and the 500 m MODIS GPP product was used as the reference dataset to evaluate the downscaled GPP results. Moreover, in order to illustrate the advantages and benefits of the proposed downscaling method, the downscaled results in this work, along with ordinary kriging downscaled results, spline downscaled results and inverse distance weighted (IDW) downscaled results, were compared to the MODIS GPP at 500 m. The results showed that (1) the GPP difference between the 500 m MODIS GPP and the proposed downscaled GPP results was primarily in the range of [−1, 1], showing that both vegetation heterogeneity factors (i.e., LAI) and topographic factors (i.e., altitude, slope and aspect) were useful for GPP downscaling; (2) the proposed downscaled results (R2 = 0.89, RMSE = 1.03) had a stronger consistency with the 500 m MODIS GPP than those of the ordinary kriging downscaled results (R2 = 0.43, RMSE = 1.36), the spline downscaled results (R2 = 0.40, RMSE = 1.50) and the IDW downscaled results (R2 = 0.42, RMSE = 1.10) for all Julian days; and (3) the inconsistency between MODIS GPP at 500 m and 1 km increased with the increase in altitude and slope. The proposed downscaling algorithm could provide a reference when considering the effects of spatial heterogeneity on carbon flux simulations and retrieving other fine resolution ecological-physiology parameters (e.g., net primary productivity and evaporation) over topographically complex terrains.