The cause is the ESP32 bug.

M5Stack Community :
M5.BtnA.wasPressed() conflicts with WiFi
https://forum.m5stack.com/topic/3262/m5-btna-waspressed-conflicts-with-wifi

This is a report of measures for the A button of M5Stack and the B button of M5Stick C to operate without permission.
ButtonA on M5Stack does not work properly : May 02, 2021

void setup(){
WiFi.begin();
WiFi.setSleep(false);
}

or

#include <driver/adc.h>
void setup(){
adc_power_acquire(); // ADC Power ON

:

blog :
ButtonA on M5Stack does not work properly : May 02, 2021
https://macsbug.wordpress.com/2021/05/02/buttona-on-m5stack-does-not-work-properly/

0_1673987001392_p.jpg