The structure, behavior and properties of gypsum mortars with different types of fine aggregates were studied. Investigation by scanning electron microscopy shows that the presence of aggregate particles significantly influences the shape and size of gypsum crystals in the interfacial transition zone. The porosity of the gypsum matrix in mortars is greater than the porosity of pure gypsum paste and the pore size distribution shifts towards smaller pores. The shape and size of crystals in the transition zone depends on the surface roughness of the aggregate particles. The grain surface roughness was measured by confocal laser scanning microscopy. The properties of mortars also depend on the grain surface roughness. The workability, setting times and thermal conductivity decrease with increasing grain surface roughness, while the strength and water vapor resistance increase.