The orthogonal least square regression (OLSR) serves as a pretty significant problem for the dimensionality reduction. Due to lack of the scale change in OLSR, the scaling term is at first introduced to OLSR to build up a novel orthogonal least square regression with optimal scaling (OLSR-OS) problem. However, OLSR-OS is still sensitive to the outliers, such that associated results could be fallacious. To strengthen the robustness of OLSR-OS, we propose an original robust OLSR-OS (ROLSR-OS) problem in 2,1-norm. To tackle a more ill-defined situation, ROLSR-OS in 2,1-norm can be further extended to ROLSR-OS in capped 2-norm. Besides, the associated ROLSR-OS methods could be derived by solving the re-weighted counterparts of ROLSR-OS problems in both norms. Moreover, the equivalence between the re-weighted counterparts and the original ROLSR-OS problems is also provided along with the convergence analysis of the proposed ROLSR-OS methods. Accordingly, both the optimal scaling and weight can be achieved automatically via the proposed ROLSR-OS approaches. Specifically, the proposed ROLSR-OS methods are self-adaptive, such that the smaller weight would be automatically assigned to the term with larger outliers to enhance the robustness. Consequently, projected clustering and modified projected clustering under the proposed ROLSR-OS problems are further investigated both theoretically and experimentally. © 2017 IEEE.