v1.0.6 of Board Manager not working **Solved**



  • It appears that the v1.0.6 version of the board manager is not working. I believe its related to switching to esptool 3.00. The specific issue I'm running into is a compile error on line 39 of the esptool.py that is installed by the v1.0.6 board version, saying "can't import serial"

    I tried to look deeper into the issue, it appears to be quite the ordeal with the esptool.py (quite a few issues listed in the script...) but it appeared the fix was to update the install to use pyserial.

    I am using the latest version of Arduino 1.8.13 on Mac OS 10.14.6 . When I downgrade to 1.0.5 (v2.6 of esptool...) all of my issues disappear. I would open an issue on github but there didnt seem to be an appropriate place. Thank you!


  • Global Moderator

    Hello, could you please try to remove M5Stack from the board manager and re-install it, we've fixed the issue last week and it should solve the problem.

    We've upgraded the library from using python2.7 to python3 which means you might not have pyserial installed in python3, to install please open terminal and type pip3 install pyserial that should install the missing pyserial on your python3 environment.

    If you get missing pip3 error (you don't have pip3 installed) do the following using terminal:

    curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
    python get-pip.py
    

    That should download and install pip using python3.



  • So I tried your recommendations as suggested:

    1. Installed pyserial. Python3 was already installed.
    'pip3 install pyserial
    Collecting pyserial
       Using cached pyserial-3.5-py2.py3-none-any.whl (90 kB)
     Installing collected packages: pyserial
      Successfully installed pyserial-3.5
    
    1. Removed 1.0.5 of M5stack from board manager, restarted arduino IDE, Installed 1.0.6, restarted IDE. Get the following error when trying to compile the factory test program:
    Traceback (most recent call last):
      File "/Users/User1/Library/Arduino15/packages/m5stack/tools/esptool_py/3.0.0/esptool.py", line 39, in <module>
        import serial
    ImportError: No module named serial
    

  • Global Moderator

    Hmm seems like your system might be using python2 by default, would you mind last thing to try:

    pip install pyserial
    

    and see if it works?



  • When I did that I got the following message:

    pip install pyserial
    Requirement already satisfied: pyserial in /usr/local/lib/python3.8/site-packages (3.5)
    

    So it appears that my system is using 3.8 already?

    I appreciate your help in trying to figure this out!


  • Global Moderator

    Hi, I see now we do have some issues with it, my recommendation is please use version 1.0.4 for now, the only difference is the ESPTool fix for BigSur, it doesn't have any improvements except that.

    could you please do one last test:

    python --version
    

    see what version do you have by default, and then run the python command and inside type import serial see if it gives you any error?

    I will dig into it and see how we can solve it completely over the next update.



  • When I run that command I get:

    python --version
    Python 2.7.16
    

    And when I try to import serial, I get the same error as with the esptool script:

    Python 2.7.16 (default, Jan 27 2020, 04:46:15) 
    [GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import serial
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ImportError: No module named serial
    

    So it appears that "pip" is symlinked to my python3 install, but "python" is still linked to the v2.7.16 default mac install.

    Thanks again for your help, will wait for an update!



  • Hello !
    I have the same issue on MacOSX Mojave 10.14.6. I updated my terminal profiles, as well as my $PATH to make sure the last version of Python is called :

    bash-3.2$ python --version
    Python 3.9.0
    bash-3.2$ python
    Python 3.9.0 (v3.9.0:9cf6752276, Oct  5 2020, 11:29:23)
    [Clang 6.0 (clang-600.0.57)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import serial
    >>> serial
    <module 'serial' from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/serial/__init__.py'>
    >>>
    

    However, I still get the following message from Arduino IDE v1.8.13 :

    Traceback (most recent call last):
      File "/Users/.../Library/Arduino15/packages/m5stack/tools/esptool_py/3.0.0/esptool.py", line 39, in <module>
        import serial
    ImportError: No module named serial
    

    I'm trying to use the new M5Paper so I cannot downgrade to 1.0.5, as the device is not referenced in it !

    Thanks for your help and your amazing work on M5Paper
    Lydéric



  • Hello,
    It seems that Arduino IDE is force-using the MacOSX official python package and not any other version installed (even if $PATH or alias in profile force to use python3).

    I succeed in solving this issue by locating the site-packages of the MacOSX python and forcing an installation of pyserial in this specified target.

    Step1 - Locate site-packages target :

    bash-3.2$ python
    Python 2.7.16 (default, Jan 27 2020, 04:46:15)
    [GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sys
    >>> print sys.path
    [''..., '/Library/Python/2.7/site-packages',...']
    >>> 
    

    Step2 - Install pyserial in the desired target

    bash-3.2$ sudo pip3 install --target /Library/Python/2.7/site-packages pyserial
    

    Step3 - verify correct installation

    bash-3.2$ python
    Python 2.7.16 (default, Jan 27 2020, 04:46:15)
    [GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import serial
    >>> serial
    <module 'serial' from '/Library/Python/2.7/site-packages/serial/__init__.py'>
    >>>
    

    Everything worked on my side after that.
    Hope it helps !
    Lydéric


  • Global Moderator

    @revta Yes, it's seems to be configuration issue in your machine, please try to do the following:

    python -m pip install pyserial
    

    See if it can solve the issue for you.



  • Just to close the loop for anyone from the future...

    Here's the steps I took to resolve this:

    1. Uninstall / Cleanup / Restore to macos Mojave default python installation. (Which is v2.7.16 as noted above.) This was a pretty intensive step. The short breadcrumb for anyone else:

    Dont touch the mac system python installs located at:
    /System/Library or
    /usr/bin

    helpful stack overflow post for uninstalling python

    But in the end, from what I could find Mojave doesn't include pip by default, and easy_install is depreciated so we had to go to step 2.

    1. brew install python Today in Dec 2020 this will install 3.9.0. ((Python 2.7 has been long depreciated))[Also note pip is currently installed with this version of python]
    2. At this point if you type python --version you will still get: 2.7.16
    3. While reading the ESP-IDF toolchain instructions, I noticed the following command:
      ln -s /usr/local/bin/python3 /usr/local/bin/python
      This setup a symlink to the python3 symlink also installed by brew. (Seems kind of hacky, but technically works...) I also had to do: ln -s /usr/local/bin/pip3 /usr/local/bin/pip which cleaned up any remaining issues making pip work. (Biggest issue being me forgetting to type pip3)
    4. After restarting my terminal, running python --version I now get: 3.9.0
    5. With my new python env working as expected, I fired up ardunio IDE and one of the Core2 examples, and went to verify to see if my issues went away... And I still had the issues above in my original post.
      6b. At this point I tried a ton of other things, most notably redoing everything by installing pyenv and trying to set the global env to 3.9, but none of those steps worked...
    6. I went back through the troubleshooting steps above, had the expected correct outcomes. Even ran the esp-tools.py directly and didnt have any issues importing serial or anything else.
    7. I believe the actual issue is the first line in the esp-tools.py:
    #!/usr/bin/env python
    

    Which I believe is what is forcing the ardunio IDE to use the default MacOS python version. (See note in #1 about this being the protected default python that we shouldnt touch.)

    1. Just as a test, I changed that line to be:
    #!/usr/bin/env python3
    

    Once I reopened Ardunio, this worked and verified without any issues.

    1. However, I feared that if I ever updated the board definitions I would forget about this change and it would all break again. So I followed @lydericc instructions above. (Thank you!) You really shouldnt do this, but it's the only way to make it "work" with everything else out of the box. Specifically the following line (Step 2 in the post above):

    sudo pip3 install --target /Library/Python/2.7/site-packages pyserial

    This installed the required serial module and fixed the remaining issues.

    I'm not sure I like the solution, but it is the only thing that has made it work. The "correct" fix would be to have esp-tool.py use the installed default python version instead of the specific one installed at /usr/bin.

    Finally, as with all things in life, there is already an xkcd for this: https://xkcd.com/1987/

    Maybe I'll open an issue with the esp-tool or Ardunio IDE and see what happens, but I dont believe this has anything to do with M5stack tools. Thanks for all who helped! (@Zontex & @lydericc )