(Part 1) How I created an autonomous vehicle as a BCA student? Story of Brainly - A vehicle with a brain!

Contents

How the idea originated

Research Selection of programming lanaguage Selection of hardware

Configuration of hardware

Programming code of Slave (Arduino)

Programming code for Master (Python)

How the idea originated

From the early days of my programming journey, I have always been someone with a passion for creating something outstanding and something big. I was always learning and always challenging myself. So, when I got informed that I was to create an "Algorithm-based project" as a course requirement for my sixth semester, different ideas started popping through my head.

First I thought, let's create a neural network-powered Android application that can communicate with its user and using NLP (Natural Language Processing), can predict users' disease and suggest to them what to do next. For this project to be successful, I figured I needed three parts i.e. 

• The program has to be able to understand what the user says
• It has to predict disease based on what the user says
• It has to prescribe the user with appropriate solution

Communication and NLP was not an issue and I was confident I could do it. But understanding disease and prescribing solutions, seemed impossible and it occurred to me that for that to be possible, I had to have a medical background. So I "dropped" the idea and started brainstorming. I had a few ideas but there were some problems regarding each idea.

Finally, when the idea of creating an autonomous vehicle came into my mind, I was sure that it was what I wanted to do. Although me and my project partner, I had no idea how we were going to achieve what we had planned, we took it upon ourselves to figure it out and do it. It was a cool challenge to ourselves and as someone who always had a huge interest in IoT (Internet Of Things), making something like this was a dream come true. So we set off in venture to create "Brainly - A Vehicle with a Brain".

Researching for the project

Thousands of articles and hundreds of YouTube videos later, I was confident enough that I could do it. I started shortlisting all the possible ways of achieving what we were planning. Now all that's left is to select the right and appropriate technology stack to make our job most effective and efficient.

Selection of programming language

Selection of programming language was quite easy as the Arduino board we were using was programmed in Arduino library which originally is a C++ library. As for the part to detect lanes and roads, it seemed like Python was our best with its huge library base. Some of the things we had to do manually were made easy by libraries like Numpy, OpenCV, Pandas, etc.

Selection of Hardware

I had a few major ideas with each having its own benefits and issues.

SN	Method	Problem & Benefit
1	Master slave connection between Arduino and Raspberry Pi Under this idea, we planned on making it so that our Arduino is hooked up to Raspberry Pi with the help of a communication interface cable. Our Raspberry Pi would have a camera, maybe an ESP32 cam, for capturing feed. The feed would be processed in the PI and it would send a signal to Arduino to control the driver motor shield.	Benefits Very low latency An effective and efficient way Problems Shortage of microcontroller boards The price of Raspberry Pi was too high for us
2	Master slave connection between Arduino and Android The other idea was to create an Android application that would have permission to capture video feeds. Analyze the video feed and send a signal to Arduino. Arduino would be hooked up to an Android device using a USB port with the help of an OTG Cable. The smartphone would directly power the Arduino board and all other hardware components.	Benefits Latency would be minimal so we would have a high response rate The camera of android could be used for feed capture so no extra camera would be needed Problems Coding the entire application with the sole purpose of controlling and automating vehicle would be time-consuming The voltage provided by the USB port of our Android was not enough to power all the components
3	Master slave connection between Arduino and Windows The plan here was to somehow transmit the camera feed from the phone to our PC, analyze that from our PC using the custom program we wrote, and again somehow send the direction the vehicle should take, wirelessly to Arduino. Sounds complicated right? At least for me, this sounded much more complicated than it actually is. But this is the approach we took in solving the problem at hand.	Benefits Minimal cost We could code in the technology stack we selected earlier Problems We had no idea where to start Huge latency issue could arise as our entire system relied on wireless communication with each other

 

So we went with the third option and created this design to showcase what our hardware would look like. The jumper wires are not shown in the picture.

 

Configuration of hardware

So let's see how I configured the hardware for the project. The components used for the project are listed below:

• Arduino UNO Mini
• HC-06 Bluetooth module
• Android-based smartphone
• Jumper wires
• TT Motors, wheels, and Chasis
• L293D Driver motor shield

Each component has its own functionality. First of all, Arduino is the main microcontroller used to control all other components. HC-06 is a Bluetooth module that will act as a receiver to receive commands from our master device. Driver motor shield is responsible for controlling TT Motors. We are using the smartphone to transfer the camera feed to the master. In our case, the feed is processed using Python code.

How did I create the software for the Arduino?

This was the most fun part of all projects, maybe because it is what I love doing. So we had two problems in hand, how to write a code to control the microcontroller board and how to write the master control code in Python. The first part was easier than I anticipated. All I had to do was write using Arduino IDE. The code was written in C++ but Arduino had some of its own side and syntax. The code for Arduino had to receive the signal from the master, process the signal, and move the vehicle based on that.

Now for receiving the signal, we just used serial communication via Bluetooth. If you do not know what serial communication is, it is basically the process of sending data from sender to receiver, one bit at a time. The code used Standard Firmata, an Arduino library that helped us in communication. The processing part was easy as all we had to do was use some conditional statements.

The most important part of the code was controlling the vehicle itself. For that, we had to control the TT Motors and the wheels attached to them, programmatically. There were 5 signals and for each signal, the following task was to be performed:

SN	Signals	What to do?
1	GO_FORWARD	Rotate all 4 wheels in a forward direction
2	GO_BACKWARD	Rotate all 4 wheels in a backward direction
3	GO_LEFT	Stop two wheels on the left and only rotate the two on the right side
4	GO_RIGHT	Stop two wheels on the right and rotate the two on the left
5	STOP	Stop rotating all 4 wheels

For controlling the motors, all we had to do was find the pins that were used to connect each motor. In our case, we were using the L293D Driver Motor shield which used digital pins 11, 3, 5, and 6. Then, after we knew which pins were connected to what motors, we controlled the power supply based on the signal.

For those who are new to this stuff, the direction of motors' rotation is based on the power being supplied to them. For example, if the motors are supplied with correct positive (+ve) and negative (-ve) power, they rotate in the forward direction, and reversing the power results in the motor rotating in the backward direction. So, GO_FORWARD and GO_BACKWARD were sorted like that. For the STOP signal, all we had to do was cut the power when we received the signal. To go left or right, we had to supply the correct +ve and -ve current to the motor on the opposite side of where we wanted to go and cut the power on the side we were traversing to.  For example, if we wanted to send a vehicle to the right, the motors on the left were to be powered and the motors on the right were to be stopped, and vice versa. With some lines of code, we were easily able to do that and now our vehicle was completely ready, and all we had to do was write control codes in Python for sending the signals.

Creating a master control system (Python)

I feel like this section requires a separate blog itself as it was the longest phase of our project. But if you are eager to know, I will keep it short here. To know more, please keep an eye out and I will update this section with a link to part 2 of this blog once I complete it. 

So the Python script consisted of these things:

1. Connection and communication system: The script would connect to the HC-06 Bluetooth module and use Serial communication.
2. Manual control system: The script would listen for any key pressed by the user and send appropriate signals to the Arduino device. Note: The vehicle was controlled using the arrow keys of the keyboard.
3.  Self-Driving System: There was a script that used various algorithms to automatically detect lane lines, traffic lights, and road signs and using computer vision and NN, send the appropriate signal to the connected car
4. Voice Control System: Using Acoustic modeling and Viberti technologies, the user could control and send the signals to the car using his voice. For example, all you had to do to send the car flying was turn on the voice control system and say "Go" or "Forward" or any other word along those lines.

This was a hugely summarized version and the actual coding process was multiple times more complicated than this explanation. So as I said earlier, I will be updating this section with a new link as soon as I finish writing the detailed version of how the code for Master was actually implemented.

Until then, keep learning, and keep growing.