v1.0.6 of Board Manager not working **Solved**

revta

It appears that the v1.0.6 version of the board manager is not working. I believe its related to switching to esptool 3.00. The specific issue I'm running into is a compile error on line 39 of the esptool.py that is installed by the v1.0.6 board version, saying "can't import serial"

I tried to look deeper into the issue, it appears to be quite the ordeal with the esptool.py (quite a few issues listed in the script...) but it appeared the fix was to update the install to use pyserial.

I am using the latest version of Arduino 1.8.13 on Mac OS 10.14.6 . When I downgrade to 1.0.5 (v2.6 of esptool...) all of my issues disappear. I would open an issue on github but there didnt seem to be an appropriate place. Thank you!

Zontex

Hello, could you please try to remove M5Stack from the board manager and re-install it, we've fixed the issue last week and it should solve the problem.

We've upgraded the library from using python2.7 to python3 which means you might not have pyserial installed in python3, to install please open terminal and type pip3 install pyserial that should install the missing pyserial on your python3 environment.

If you get missing pip3 error (you don't have pip3 installed) do the following using terminal:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py

That should download and install pip using python3.

revta

So I tried your recommendations as suggested:

Installed pyserial. Python3 was already installed.

'pip3 install pyserial
Collecting pyserial
   Using cached pyserial-3.5-py2.py3-none-any.whl (90 kB)
 Installing collected packages: pyserial
  Successfully installed pyserial-3.5

Removed 1.0.5 of M5stack from board manager, restarted arduino IDE, Installed 1.0.6, restarted IDE. Get the following error when trying to compile the factory test program:

Traceback (most recent call last):
  File "/Users/User1/Library/Arduino15/packages/m5stack/tools/esptool_py/3.0.0/esptool.py", line 39, in <module>
    import serial
ImportError: No module named serial

Zontex

Hmm seems like your system might be using python2 by default, would you mind last thing to try:

pip install pyserial

and see if it works?

revta

When I did that I got the following message:

pip install pyserial
Requirement already satisfied: pyserial in /usr/local/lib/python3.8/site-packages (3.5)

So it appears that my system is using 3.8 already?

I appreciate your help in trying to figure this out!

Zontex

Hi, I see now we do have some issues with it, my recommendation is please use version 1.0.4 for now, the only difference is the ESPTool fix for BigSur, it doesn't have any improvements except that.

could you please do one last test:

python --version

see what version do you have by default, and then run the python command and inside type import serial see if it gives you any error?

I will dig into it and see how we can solve it completely over the next update.

revta

When I run that command I get:

python --version
Python 2.7.16

And when I try to import serial, I get the same error as with the esptool script:

Python 2.7.16 (default, Jan 27 2020, 04:46:15) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import serial
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named serial

So it appears that "pip" is symlinked to my python3 install, but "python" is still linked to the v2.7.16 default mac install.

Thanks again for your help, will wait for an update!

lydericc

Hello !
I have the same issue on MacOSX Mojave 10.14.6. I updated my terminal profiles, as well as my $PATH to make sure the last version of Python is called :

bash-3.2$ python --version
Python 3.9.0
bash-3.2$ python
Python 3.9.0 (v3.9.0:9cf6752276, Oct  5 2020, 11:29:23)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import serial
>>> serial
<module 'serial' from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/serial/__init__.py'>
>>>

However, I still get the following message from Arduino IDE v1.8.13 :

Traceback (most recent call last):
  File "/Users/.../Library/Arduino15/packages/m5stack/tools/esptool_py/3.0.0/esptool.py", line 39, in <module>
    import serial
ImportError: No module named serial

I'm trying to use the new M5Paper so I cannot downgrade to 1.0.5, as the device is not referenced in it !

Thanks for your help and your amazing work on M5Paper
Lydéric

lydericc

Hello,
It seems that Arduino IDE is force-using the MacOSX official python package and not any other version installed (even if $PATH or alias in profile force to use python3).

I succeed in solving this issue by locating the site-packages of the MacOSX python and forcing an installation of pyserial in this specified target.

Step1 - Locate site-packages target :

bash-3.2$ python
Python 2.7.16 (default, Jan 27 2020, 04:46:15)
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print sys.path
[''..., '/Library/Python/2.7/site-packages',...']
>>>

Step2 - Install pyserial in the desired target

bash-3.2$ sudo pip3 install --target /Library/Python/2.7/site-packages pyserial

Step3 - verify correct installation

bash-3.2$ python
Python 2.7.16 (default, Jan 27 2020, 04:46:15)
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import serial
>>> serial
<module 'serial' from '/Library/Python/2.7/site-packages/serial/__init__.py'>
>>>

Everything worked on my side after that.
Hope it helps !
Lydéric

Zontex

@revta Yes, it's seems to be configuration issue in your machine, please try to do the following:

python -m pip install pyserial

See if it can solve the issue for you.

revta

Just to close the loop for anyone from the future...

Here's the steps I took to resolve this:

Uninstall / Cleanup / Restore to macos Mojave default python installation. (Which is v2.7.16 as noted above.) This was a pretty intensive step. The short breadcrumb for anyone else:

Dont touch the mac system python installs located at:
/System/Library or
/usr/bin

helpful stack overflow post for uninstalling python

But in the end, from what I could find Mojave doesn't include pip by default, and easy_install is depreciated so we had to go to step 2.

brew install python Today in Dec 2020 this will install 3.9.0. ((Python 2.7 has been long depreciated))[Also note pip is currently installed with this version of python]
At this point if you type python --version you will still get: 2.7.16
While reading the ESP-IDF toolchain instructions, I noticed the following command:
ln -s /usr/local/bin/python3 /usr/local/bin/python
This setup a symlink to the python3 symlink also installed by brew. (Seems kind of hacky, but technically works...) I also had to do: ln -s /usr/local/bin/pip3 /usr/local/bin/pip which cleaned up any remaining issues making pip work. (Biggest issue being me forgetting to type pip3)
After restarting my terminal, running python --version I now get: 3.9.0
With my new python env working as expected, I fired up ardunio IDE and one of the Core2 examples, and went to verify to see if my issues went away... And I still had the issues above in my original post.
6b. At this point I tried a ton of other things, most notably redoing everything by installing pyenv and trying to set the global env to 3.9, but none of those steps worked...
I went back through the troubleshooting steps above, had the expected correct outcomes. Even ran the esp-tools.py directly and didnt have any issues importing serial or anything else.
I believe the actual issue is the first line in the esp-tools.py:

#!/usr/bin/env python

Which I believe is what is forcing the ardunio IDE to use the default MacOS python version. (See note in #1 about this being the protected default python that we shouldnt touch.)
9. Just as a test, I changed that line to be:

#!/usr/bin/env python3

Once I reopened Ardunio, this worked and verified without any issues.

However, I feared that if I ever updated the board definitions I would forget about this change and it would all break again. So I followed @lydericc instructions above. (Thank you!) You really shouldnt do this, but it's the only way to make it "work" with everything else out of the box. Specifically the following line (Step 2 in the post above):

sudo pip3 install --target /Library/Python/2.7/site-packages pyserial

This installed the required serial module and fixed the remaining issues.

I'm not sure I like the solution, but it is the only thing that has made it work. The "correct" fix would be to have esp-tool.py use the installed default python version instead of the specific one installed at /usr/bin.

Finally, as with all things in life, there is already an xkcd for this: https://xkcd.com/1987/

Maybe I'll open an issue with the esp-tool or Ardunio IDE and see what happens, but I dont believe this has anything to do with M5stack tools. Thanks for all who helped! (@Zontex & @lydericc )