Friction has a direct relation with the energy efficiency and environmental cleanliness in all moving mechanical systems. To develop low friction coatings is extremely beneficial for preserving not only our limited energy resources but also the earth’s environment. This study proposes a new design for low friction carbon-based nanocomposite coatings by tailoring the microstructure and phase segregation, and thereby it contributes to better controlling the mechanical and tribological properties. Experimental findings and theoretical calculations reveal that high-hardness (18.2 GPa), high-adhesion strength (28 N) as well as low-internal stress (−0.8 GPa) can be achieved by a nanocrystallite/amorphous microstructure architecture for the nc-WC/a-C(Al) carbon-based nanocomposite coating; in particular low friction (∼0.05) can be acquired by creating a strong thermodynamic driving force to promote phase segregation of graphitic carbon from the a-C structure so as to form a low shear strength graphitic tribo-layer on the friction contact surfaces. This design concept is general and has been successfully employed to fabricate a wide class of low friction carbon-based nanocomposite coatings.