The last letter to disappear was the long S (ſ), which looks too much like the F and confused readers. Anybody reading a manuscript written before the middle of the 18th century will recognize the long S. Noah Webster used it in his preface to his 1806 dictionary.
However, two of these antique letters should have been retained. The modern digraph TH has two sounds, the hard th heard in breath, and the soft th heard in breathe. We must use two letters (T and H) to make the sounds, yet, much to the confusion of little children and adults learning English, those same two letters make different sounds.
The thorn (Þ) had the hard th sound we hear in modern words like thing, think, and thigh. The runic eth (ð) appears to have had the soft th sound we hear in they and that. A separate letter for each of the th sounds seems more logical than using two letters to make two different sounds.
Consider also the Greek theta (Θ). This single letter has the sound of th, and while every scholar knew that it existed, rarely, if ever, did they use it. If the theta had been brought into English for the th sound, it might or might not have improved our reading and spelling. But we’ll never know because as England rapidly became part of Western Europe, the English alphabet aligned itself to match the alphabet used on the continent, and all those runic letters disappeared.
On the one hand, we may feel sad to see them go, because they might find them useful. But on the other hand, where would we put them on our qwerty key boards?