Posts made by kkkkkk

Hi everyone, I want to integrate this development board into M5Stack-Basic. Is it possible? In other words, M5Stack has devices that support voice recognition.

Introduction:

In the movie "Iron Man, " we see Tony Stark communicating with the AI Jarvis while building equipment. Tony describes to Jarvis the parts he needs, and Jarvis controls robotic arms to assist Tony in completing the task. With the development of technology today, this kind of implementation is only a matter of time. Therefore, I decided to attempt to implement this function myself, using voice control to operate a robotic arm and achieve a simple application of artificial intelligence.

I was solely responsible for connecting and controlling the robotic arm and voice development board, triggering the movement of the robotic arm through specific commands. This was a challenging project because voice control technology involves multiple aspects, such as voice recognition, signal processing, and robot control, requiring me to learn and understand related knowledge. Through this article, I hope to share my project implementation process, achievements, and lessons learned, in order to inspire and assist fellow students working in the fields of robotics and voice control.

To implement this project, I first needed to solve two main problems: how to process voice signals and how to control the robotic arm.

Processing Voice Signals:

Steps for processing voice signals:

● Hardware selection: Choose a suitable development board (Arduino, Raspberry Pi, etc.).

● Microphone signal processing: Select a suitable microphone for collection, and convert the collected voice signal into a digital signal through an analog-to-digital converter.

● Signal processing: The collected signal is processed through noise reduction, filtering, etc. Useful features are extracted, and the feature signal is processed using a speech recognition algorithm to output a digital signal to the computer.

Controlling the Robotic Arm:
● Use a simple and easy-to-operate robotic arm.

The above is the way to solve the two main problems. However, after research, developing a development board that can process voice signals requires a large amount of work! Fortunately, there are integrated voice development boards on the market now, and we can directly purchase one to use.

HLK-V20 Voice Development Board

The HLK-V20 development board is a sophisticated board with two IO interfaces - one for voice output and the other for voice input. It can be connected to a computer through a serial port.

MyCobot 280 M5Stack 2023 Robotic Arm

For the robotic arm, I have chosen the popular desktop robotic arm, myCobot 280 M5Stack 2023, developed by Elephant Robotics. This robotic arm has recently undergone a significant software update, achieving a 20ms serial communication speed and offering rich interfaces compatible with popular programming languages such as Python and C++. I chose this robotic arm for several reasons, including its compact size, rich Python API interfaces (which I am skilled in programming), and ease of installation and use.

Development Environment

Programming language: Python 3.7+

Platform: Windows 11

Python libraries used: serial, pymycobot, time.

Development of the project

In the development of the project, there are four main steps involved:

● Speech input

● The development board retrieves the speech information and transfers it to the PC

● The PC processes the received information

● Sends motion control commands to the robotic arm for motion

Speech Recognition Development

To observe how the development board transfers data and what type of data it returns, the serial library in Python is used to operate the serial port. The library has methods to open, close, read, and write data from the serial port.

'''
For instance, the serial.Serial() method is used to open the serial port and connect it to the PC, while the serial.read() method reads the data transmitted from the serial port.
'''
import serial
s = serial.Serial("COM9",115200)
r = s.read()
while True:
print(r)

Here is the data that get from read().
By analyzing the data transmitted from the development board, the wake-up word of the development board was identified as 'wakeup_uni', while the keyword 'uArTcp' indicates the start of the next command.

It was discovered that the received data is concatenated together, and each time a command is inputted, it is concatenated with the previous one.

It was discovered that the received data is concatenated together, and each time a command is inputted, it is concatenated with the previous one. To check whether the development board has received the speech input, an if a in b: statement was used, with the input commands and their corresponding keywords being previously prepared. For instance, the wake-up word was set as "wakeup_uni", and 4 prompt words were set for testing purposes.

#set prompt word
hi_mycobot = b'openled'
dancing = b'zengjialiangdu'
nod_head = b'jianxiaoliangdu'
go_back = b'closeled'
# run frame
while True:
r += s.read()
if wake_up in r:
print("wake")
wake = True
r = b''
if wake and hi_mycobot in r:
print("Hi myCobot")
r = b''
if wake and dancing in r:
print("dancing")
r = b''
if wake and nod_head in r:
print('nod your head')
r = b''
if wake and go_back in r:
print('Go back')
r = b''
if wake and _exit in r:
print("exit")
r = b''
# print(r)s

This is the result of the test
Robotic Arm Control Development
The pymycobot library is a robotic arm API call interface library developed by Elephant Robotics. The specific usage method is on Gitbook (by Elephant Robotics).

# import library
from pymycobot import Mycobot
import time
# Create an instance object for later use
mc = Mycobot('COM9',115200)
#Control the mechanical arm to move at a speed of 70 according to the angle in the list，send_angles([angles],speed)
mc.send_angles([0.87,(-50.44),47.28,0.35,(-0.43),(-0.26)],70)
# Execute the next command after a delay of x seconds (the movement of the robotic arm takes time)
time.sleep(x)

Next, combine the two parts of the code together.

import serial
from pymycobot import MyCobot
import time
s = serial.Serial("COM6",115200)
mc = MyCobot('COM3',115200)
r = b''
wake_up = b'wakeup_uni'
_exit = b'exitUni'
hi_mycobot = b'openled'
dancing = b'zengjialiangdu'
nod_head = b'jianxiaoliangdu'
go_back = b'closeled'
wake = False
while True:
r += s.read()
if wake_up in r:
print("wake")
wake = True
r = b''
if wake and hi_mycobot in r:
print("Hi myCobot")
# say hi shake with you
mc.send_angles([0.87,(-50.44),47.28,0.35,(-0.43),(-0.26)],70)
time.sleep(1)
for count in range(3):
mc.send_angle(1,30,80)
time.sleep(1)
mc.send_angle(1,(-30),80)
time.sleep(1)
mc.send_angles([0,0,0,0,0,0],70)
r = b''
if wake and dancing in r:
print("dancing")
mc.send_angles([0,0,0,0,0,0],80)
time.sleep(1)
for count in range(2):
mc.send_angles([(-0.17),(-94.3),118.91,(-39.9),59.32,(-0.52)],80)
time.sleep(1.2)
mc.send_angles([67.85,(-3.42),(-116.98),106.52,23.11,(-0.52)],80)
time.sleep(1.7)
mc.send_angles([(-38.14),(-115.04),116.63,69.69,3.25,(-11.6)],80)
time.sleep(1.7)
mc.send_angles([2.72,(-26.19),140.27,(-110.74),(-6.15),(-11.25)],80)
time.sleep(1)
mc.send_angles([0,0,0,0,0,0],80)
r = b''
if wake and nod_head in r:
print('nod your head')
mc.send_angles([0,0,0,0,0,0],70)
time.sleep(1)
mc.send_angles([3.07,(-86.3),75.32,11.86,2.72,(-0.35)],70)
time.sleep(0.5)
for count in range(4):
mc.send_angle(4,13,70)
time.sleep(0.5)
mc.send_angle(4,(-60),70)
time.sleep(1)
mc.send_angle(4,13,70)
time.sleep(0.5)
mc.send_angles([0,0,0,0,0,0],70)
r = b''
if wake and go_back in r:
print('Go back')
mc.send_angles([12.83,(-138.95),156.09,(-12.3),(-12.91),35.41],70)
r = b''
if wake and _exit in r:
print("exit")
r = b''

Here is a video to show how it run.
https://youtu.be/X16pPnVCsHo
Experience and Lessons Learned
During the process of completing this project, I gained a lot of valuable experience and insights. Firstly, I realized that the completion of a project not only depends on the mastery of technology, but also requires prior research and understanding. When I was studying the production of a speech recognition development board, I found that the workload was very large and the implementation would be very difficult. However, there are already many mature technologies and tools available on the market. We only need to choose a reasonable combination and integration. Secondly, the successful completion of a project requires a clear framework and sufficient preparation. In this project, I needed to understand speech recognition algorithms, mechanical arm design and control technology, and master the use methods and performance characteristics of hardware equipment and software development tools.

In this project, I successfully combined speech recognition technology with mechanical arm control technology to achieve voice control of the mechanical arm. Although the scale of this project is relatively small, it represents the application and development trend of artificial intelligence technology in real life. Although this is only a prototype of an AI project, its completion is a valuable experience for me. In future research, I will continue to explore more related information to further improve this project. If you have any good ideas, please leave a comment below.

Hi, how can I get the DSee-65X's information?

@ajb2k3 Thanks!I read some of the articles they posted that provided some help.

I have recently started preparing for my graduate design and have chosen a topic in the direction of a small six-axis robotic arm. While searching for information, I saw a video of a YouTuber who made a robotic arm by himself and make the robotic arm complete tasks like a human hand. I was surprised and impressed by this video, which provides me with more inspiration for robotic arm learning.

When I searched for "robotic arm," most of the search results were industrial robotic arms, which were unsuitable for my learning purpose. What I was looking for was a small robotic arm for educational use. And my requirements were:

• It can be easily installed.

• High quality with stable movement

• Sufficient information and tutorials that help me in programming

After comparing many products, finally I selected the mechArm 270-M5Stack, a small six-axis desktop robotic arm from Elephant Robotics. The mechArm 270-M5 is a multifunctional robotic arm with six degrees of freedom in an industrial design and is based on the M5Stack-ESP32. It supports connecting LEGO firmware at the end of the arm and can be equipped with many accessories, such as a suction pump and gripper.

Also, I got a lot of information about mechArm 270 and many tutorials for programming the robot from Elephant Robotics. Supporting multiple systems and programming languages is also an advantage that attracts me
The main purpose of this article is to share my feelings and problems in this learning process. I tried to control the mechArm in ROS, the Robot Operating System, which was well-developed as a good choice in robotics learning. Here is my attempt using the mechArm companion material in ROS.

（Gitbook from Elephant Robotics.）
The environment setup and cases are written in detail in Gitbook, and I wanted to follow the steps on it. When I was installing ROS, it always showed download unsuccessfully, and one solution was changing the download source. Gitbook provided an Ubuntu system with an environment that I could operate directly.

Here is was I did use the gitbook guidance:
First, I used USB Type-C to connect the mechArm to the computer.
Making sure the connection was successful, I entered the command to check.

ls /dev/tty* //Checking the PC serial connection

When I saw the '/dev/ttyACM0', I knew I had made it.
Next, I used the function described in the Gitbook, which functions as a slider to control the robotic arm.
In ROS, the Elephant Robotics team has already built the mechArm model and the slider control topic.I only took two commands to make it work.

When I finished running it, I found that my robotic arm did not follow the control slider movement. A 1:1 virtual scene is created in ROS, so another script has to be run to control the robotic arm.
I've made a video to document it.
https://www.youtube.com/watch?v=ESRLthJKoyI
（The shaking of the mechArm in the video is due to the lack of a fixed base, not a problem with the arm.）
There is also model following, keyboard control, and GUI control.
Slider control, keyboard control, and GUI control are all features of the single-joint control robotic arm, but it is MoveIt that I am most excited about, with its ability to perform motion planning, forward and reverse kinematics, 3D perception, and collision detection.
When I drag the axis at the end of the mechArm, the mechArm follows the overall movement. MoveIt can generate random poses automatically.
I also made a video recording.
https://www.youtube.com/watch?v=EZa6Vek00VQ

My thoughts and ideas:

The configuration of MoveIt can be complicated for those who are new to it and don't know where to start.

When I used mechArm, MoveIt was configured and can be used directly, which brings great convenience for programming.
I saw the configuration information of mechArm in MoveIt, which brought me great convenience.
The supporting information and data in mechArm is an attractive point for me. It is very useful and helpful for the individual maker who just starting robotics learning like me. Next, I will try to complete the MoveIt configuration of mechArm independently to start learning.
If you like this article, please give me a like or leave a comment to support it. Thank you!

I also have a myCobot and look forward to you sharing more of your experiences and sharing them together sometime!

Looks good and would like to ask a few questions.
1 Is it capable of painting any pattern?
2 How wide a range can it work? Can it draw larger patterns?
3 Is it possible to share the code?

I have one myCobot 320, and gonna buy a new one in black

What is the auxiliary control of this mechArm?

Good article, I will buy this kit