Recently, transformer-based models has shown promising results on miscellaneous computer vision tasks. However, its high computation cost makes it neither practical to deploy on mobile devices, nor economic to compute on servers. In this paper, we propose two effective quantization schemes for reducing the memory usage and computation consumption of vision transformers. First, we develop an approximation-based Post-training Quantization (PTQ) approach which optimizes for a set of quantization scaling factors that minimize quantization errors. Moreover, we introduce a learning-based Quantization-aware Training (QAT) approach that allows for model finetuning after inserting quantization operations to restore accuracy. Furthermore, we reveal the complementary effects of learning-based approach and approximation-based approach in QAT and propose an effective strategy for the initialization of quantization parameters. We evaluate our approaches on ImageNet for different vision transformer models. Our quantization algorithms outperform the previous state-of-art approaches for both post-training quantization and quantization-aware training benchmark. With weights and activations in vision transformer quantized to 8-bit integers, we obtain a ×4 compression rate of model parameters with an accuracy drop of less than 0.2% for models of various scales.