We investigate the form of mathematical structuralism that acknowledges the existence of structures and their distinctive structural elements. This form of structuralism has been subject to criticisms recently, and our view is that the problems raised are resolved by proper, mathematics-free theoretical foundations. Starting with an axiomatic theory of abstract objects, we identify a mathematical structure as an abstract object encoding the truths of a mathematical theory. From such foundations, we derive consequences that address the main questions and issues that have arisen. Namely, elements of different structures are different. A structure and its elements ontologically depend on each other. There are no haecceities and each element of a structure must be discernible within the theory. These consequences are not developed piecemeal but rather follow from our definitions of basic structuralist concepts.