The Event Horizon Telescope (EHT) has recently delivered the first resolved images of M87*, the supermassive black hole in the center of the M87 galaxy. These images were produced using 230 GHz observations performed in 2017 April. Additional observations are required to investigate the persistence of the primary image feature-a ring with azimuthal brightness asymmetry-and to quantify the image variability on event horizon scales. To address this need, we analyze M87* data collected with prototype EHT arrays in 2009, 2011, 2012, and 2013. While these observations do not contain enough information to produce images, they are sufficient to constrain simple geometric models. We develop a modeling approach based on the framework utilized for the 2017 EHT data analysis and validate our procedures using synthetic data. Applying the same approach to the observational data sets, we find the M87* morphology in 2009-2017 to be consistent with a persistent asymmetric ring of similar to 40 mu as diameter. The position angle of the peak intensity varies in time. In particular, we find a significant difference between the position angle measured in 2013 and 2017. These variations are in broad agreement with predictions of a subset of general relativistic magnetohydrodynamic simulations. We show that quantifying the variability across multiple observational epochs has the potential to constrain the physical properties of the source, such as the accretion state or the black hole spin.