That sounds frustrating! Since it's been an issue since purchase, it might be a hardware defect in the IT8951 controller. Try a 'nuclear reset'。 Disconnect the battery for 30 mins + hold power button to drain residual charge, then reflash.
I'm looking forward to hearing about the new version of your EPub Reader for M5paper1.1. I specifically registered here to find out how things are going with the port and are you planning to add a new version to M5burner?