My answer (slightly edited) to a question on Quora:

Avoiding circular reasoning is a result of education, training, upbringing; mathematicians avoid circular reasoning because if the did not, they would never become mathematicians. To compare, there are certain things which every driver never does: for example, pressing accelerator and brakes simultaneously — for otherwise he/she would have difficulty in passing the driving test. Conveniently, the accelerator pedal and the brakes pedal are usually positioned in a car in such a way that it makes it difficult to press the accelerator with the left foot, and even harder to press the accelerator and brakes (with the right foot) simultaneously.

Perhaps it needs to be explained that mathematicians not just remember a particular theorem (actually, very frequently only an idea of the theorem, without technical details – they know that they can recover details when they need them), but, crucially, they care about, and keep in order, relations between theorems; they keep in their minds not just facts, but relations and analogies between facts, and, moreover, analogies between analogies. They sustain in their minds a multidimensional image of a theory, a complex, strongly interlinked, hierarchically built and dynamically developing system. This is like a living organism; to keep it alive, one has to follow certain rules of mental hygiene; avoidance of circular reasoning is perhaps one of the most important.

Here lies one of the principal contradiction of mathematics teaching: too frequently, teachers find that it easier to make students to memorise and mechanically use individual mathematical facts rather than opening to them the rich, vibrant, full of colours world of connections between these facts (and, to say the truth, the teachers themselves are frequently unaware about the existence of this world). Not surprisingly, students frequently have difficulties with circular arguments because it simply does not matter not only for them, but to their teachers, too, which comes first: statement A or statement B.

Students could be even more confused when they encounter completely legal circular proofs of equivalence of several different definitions (or characterisations) of the same concept or object. When writing this answer, I have checked lecture notes of the my undergraduate course of linear algebra: it contains 19 equivalent characterisations of invertible matrices, and I was able to produce, on the spot, two more. It is a kind of algebraic merry-go-round, and a deeper structural understanding is needed to avoid a dizzy spell.