You cannot compare a disorderly mob of miners and a standing army. If you are facing a group of levied poorly trained and equipped infantrymen, you can charge them easily. Half of them will start running away even before the horses reach them and the rest will not be able to create tight enough formation to stop them and also start running away (you cannot outrun a horse, but you can outrun your comrades whose bodies will slow the horses down). But if you are facing a disciplined group of well-trained pikemen, you (and your horse) would be charging to your death. Pikemen formations have a weakness though - they are hard to turn around (which is why the phalanx was replaced by the legion which is much more mobile…but that was before the invention of stirrups and plate armour which made horsemen much deadlier and so there was a need for pikemen again). If you can encircle them and attack from the back, then that is another way of using horse charges.
I guess that either cataphracts (originally Parthian, later Eastern Roman or Sassanid and probably used by some Arabians as well) or especially later their late middle ages equivalent (with full plate covering the rider and the horse) could even successfully charge a solid group of men, but again, not pikemen (or a phalanx which is essentially the same thing).
Cavalary was mostly used for killing the routing enemy and for flanking (and then you have eastern horse archers but that is a whole different category). This is actually when most casualties were suffered. Real battles were nothing like hollywood movies where you see a battle formation very quickly changing into man to man combat. It looks more dramatic and is also cheaper to film (in a scene you don’t have hundreds of soldiers, but a couple at best), but it is not real at all. A typical battle was about breaking the enemy formation…after which bloodshed in the ranks of the losers was delivered by the cavalary. Before that moment the losses on either side were minimal though. Real battles were more about discipline and endurance than about swordswinging skills because a steadily advancing orderly formation always beats loose groups of essentially individually fighting men as often demonstrated by Roman army (while it still was orderly and disciplined, i.e. not so much towards the end of the Western empire) or the Macedonian armies under Alexander and the diadochi who often fought against much less disciplined foes (who otherwise usually did not have worse weapons, fighting skills or bravery).
A good example of what a real battle actually looked like is the first scene in Rome (the HBO/BBC TV Series) where the soldiers hold a line and are actually ordered by the centurion to keep it like that instead of trying to be heroes on their own. The later battles in that series are sadly more along the lines of typical movie depiction (or not present at all for budget reasons).
