Defects are ubiquitous in materials. In high-temperature superconductors (HTS), certain defects play an important role; by pinning quantized vortices in the presence of magnetic field, they enable dissipationless transport of high current densities. Therefore, determining the atomic structure of defects as well as understanding how they behave and interact is critical to control the physical properties of HTS. This chapter presents an in-depth look into the complex microstructure of YBa2Cu3O7−x, a paradigmatic HTS, at different length scales using aberration-corrected scanning transmission electron microscopy (STEM). Furthermore, a synergistic combination of aberration-corrected STEM imaging, electron energy loss spectroscopy, X-ray magnetic circular dichroism, and density-functional-theory calculations have recently revealed point defects, such as individual vacancies and complex vacancy clusters, which affect the host crystal structure on a single unit-cell level. One such defect consisting of a complex of copper and oxygen vacancies is also shown to induce dilute ferromagnetism in YBCO HTS, which opens a playground to study the interaction between the two highly antagonistic phenomena by atomic-scale control over these defects.