According to PHB 122, in the section entitled Alignment in the Multiverse, for most creatures, "alignment is a moral choice." Myth says good-aligned gods created the races that are free to choose to be evil (they have free will), while evil-aligned gods created the races that are very inclined towards evil, since they were born to serve the evil god in the first place.
However, there are some types of creatures — like devils — which embody an alignment in its essence:
Alignment in the Multiverse, PHB 122
A devil does not choose to be lawful evil, and it doesn't tend toward lawful evil, but rather it is lawful evil in its essence. If it somehow ceased to be lawful evil, it would cease to be a devil.
My first thought was to not take this reading literally. The text could have meant that a neutral evil devil "isn't really a devil," but really, it's still a devil. It's a bit like imagining a guitarist who loses both hands. The loss of some part of a person's essence makes them not the same person anymore.
However, keeping my attention on it, I'm finding this text more and more difficult to interpret. If a devil "is lawful evil in its essence" then what does it mean when "it would cease to be a devil"? What happens to a devil when its alignment is forcibly changed?
There are ways to forcibly change someone's alignment, and a devil can be subject to some of them.
Best Answer
Alignment is to a devil what matter is to a creature of the Prime Material.
PHB p. 122
You might want to start your investigation of this question with a quick read on Plato and his philosophy of Ideals. Plato posited that everything we see in this world is just a shadow of a world of "ideals". When we see a dog, we see a lump of flesh attached to some bones and wrapped in fur, more or less like lots of other lumps that we also call "dogs". Plato said that since we can recognize all these peculiar lumps as "dogs", then somewhere there must be an "ideal dog" - a single creature that completely embodies all things doggy.
The DnD multiverse embraces this idea and places these ideals in the Outer Planes. The game no longer has all the summon spells it once did, but when it did, the implied mechanism was that whatever beast was summoned was not any particular beast but the Ideal of that beast. (And, yes, you read that right - the Ideal Dog is not a particular dog, it is all dogs - even those that never existed.)
Of course, to exist on the Prime Material that Ideal had to cloak itself in the stuff of that plane. Those of us made of ordinary earth can not properly perceive or interact with Ideals in their true form. This is why we have the concept of Avatars and why fiends who die on the Prime Material are not truly killed. You can't kill an Ideal through material weapons. If it is possible to truly kill an Ideal, it must be by subverting the very concept that the Ideal represents.
D&D 5e doesn't get very deep into this set of ideas, but there are a few hints here and there. You have stumbled into one of the most direct examples.
The Outer Planes are ideal manifestations of Law, Chaos, Good and Evil. Alignment is the stuff they are made of, as surely as the Prime Material is made of earth, water, air and heat. (DMG p. 59.)
All devils are made of that same material. When a devil visits the Prime Material, he manifests with a material body so he can be compatible with our elements, but this is only a "cloak" over his true nature. If you kill him, you are only rending this "cloak" - you can't touch his true nature with mere material weapons.
Finally, to heart of the question.
If you change a devil's alignment, you are literally changing the stuff of which he is made. Whether his appearance changes is subject to DM's discretion - the books don't say. But in all important ways, he will change into the celestial or fiend appropriate to the new alignment. There is at least one interesting edge case: the Neutral alignment. Perhaps a good follow-up question for this forum would be, "Is there an Ideal manifestation of Neutrality in the Outer Planes?"