The activity of polycrystalline thin film photoelectrodes is impacted by local variations of the material properties due to the exposure of different crystal facets and the presence of grain/domain boundaries. Here a multi‐modal approach is applied to correlate nanoscale heterogeneities in chemical composition and electronic structure with nanoscale morphology in polycrystalline Mo‐BiVO4. By using scanning transmission X‐ray microscopy, the characteristic structure of polycrystalline film is used to disentangle the different X‐ray absorption spectra corresponding to grain centers and grain boundaries. Comparing both spectra reveals phase segregation of V2O5 at grain boundaries of Mo‐BiVO4 thin films, which is further supported by X‐ray photoelectron spectroscopy and many‐body density functional theory calculations. Theoretical calculations also enable to predict the X‐ray absorption spectral fingerprint of polarons in Mo‐BiVO4. After photo‐electrochemical operation, the degraded Mo‐BiVO4 films show similar grain center and grain boundary spectra indicating V2O5 dissolution in the course of the reaction. Overall, these findings provide valuable insights into the degradation mechanism and the impact of material heterogeneities on the material performance and stability of polycrystalline photoelectrodes.