SURVEILLANCE ROBOT
Submitted by:
CHAUDHARI PINAKINKUMAR R. (140150109008)
GAJJAR DARSH A. (140150109030)
GAMIT PRAGNESHKUMAR J. (140150109032)
KUBAVAT YASH B. (140150109052)
NIKUNJKUMAR CHAUDHARI (140150109061)
Under the Guidance of
Prof. R. D. BHAGIYA,
Asst. Prof., Electrical Engineering Department,
Government Engineering College, Bhuj
A Project ort Submitted to
Gujarat Technological University
In fulfilment of the requirements of
The Degree of Bachelor of Engineering
In Electrical Engineering
April – 2018
center22987000
Government Engineering College, Bhuj
Near Uma Nagar, New Ravalwadi Relocation Site,
Bhuj, Dist. – Kachchh – 370001, Gujarat
Government Engineering College, Bhuj
Electrical Engineering Department
2018
CERTIFICATE
This is to certify that the project entitled “SURVEILLANCE ROBOT” has been carried out by following students under my guidance in fulfilment of the degree of Bachelor of Engineering in ELECTRICAL (8th Semester) of Gujarat Technological University, Ahmedabad during the academic year 2017-18.
Sr. No. Name of Students Enrolment No.
1. Chaudhari Pinakinkumar R. 140150109008
2. Gajjar Darsh A. 140150109030
3. Gamit Pragneshkumar J. 140150109032
4. Kubavat Yash B. 140150109052
5. Nikunjkumar Chaudhari 140150109061
Guided By: DR. B. N. SUTHAR
Prof. R. D. BHAGIYAHead of Department
Contents
TOC o “1-3” h z u ACKNOWLEDGEMENT PAGEREF _Toc512434540 h IGTU COMPLETION CERITIFICATE PAGEREF _Toc512434541 h IIPLAGIARISM REPORT PAGEREF _Toc512434542 h IIIABSTRACT PAGEREF _Toc512434543 h IVChapter 1: INTRODUCTION PAGEREF _Toc512434544 h 11.1Aim and Objective PAGEREF _Toc512434545 h 11.2Introduction PAGEREF _Toc512434546 h 21.2.1 Surveillance System PAGEREF _Toc512434547 h 21.2.2 Introduction to robots PAGEREF _Toc512434548 h 21.2.3 Surveillance Robot PAGEREF _Toc512434549 h 41.2.4 Why 4-legged Robot is Selected? PAGEREF _Toc512434550 h 5Chapter 2: LITERATURE SURVEY PAGEREF _Toc512434551 h 62.1Inverse Kinematics PAGEREF _Toc512434552 h 62.2 Forward Kinematics PAGEREF _Toc512434553 h 82.3 Centre of Gravity PAGEREF _Toc512434554 h 8Chapter 3: USER DEFINED PROBLEM FORMULATION & DESIGN METHHOLOGY PAGEREF _Toc512434555 h 93.1 What is the practical function of the design? PAGEREF _Toc512434556 h 93.2 Walking PAGEREF _Toc512434557 h 103.2.1 Basics PAGEREF _Toc512434558 h 103.2.2Walking Flowchart & Algorithm PAGEREF _Toc512434559 h 113.3Design of Shape PAGEREF _Toc512434560 h 173.4Video Streaming and Object Recognition/Tracking PAGEREF _Toc512434561 h 19Chapter 4: USER DEFINED PROBLEM SOLUTION/ HARDWARE COMPONENTS DESCRIPTION PAGEREF _Toc512434562 h 20Chapter 5: HARDWARE IMPLEMENTATION/ SOFTWARE DETAILS WITH PROGRAM PAGEREF _Toc512434563 h 295.1Arduino Programming for Dynamic Walking Gait PAGEREF _Toc512434564 h 295.2Arduino Program Explanation PAGEREF _Toc512434565 h 405.3Object Recognition using Deep Neural Networks and OpenCV Library of Trained Models and Blobs to Image Conversion PAGEREF _Toc512434566 h 465.4 Object Detection and Mask Creation as per colour HSV value/ Python and Tkinter based GUI version of Video Stream Facilitates with Taking Screenshot PAGEREF _Toc512434567 h 50Chapter 6: SUMMARY OF RESULTS PAGEREF _Toc512434568 h 536.1 Advantages PAGEREF _Toc512434569 h 536.2 Disadvantages PAGEREF _Toc512434570 h 53CONCLUSION PAGEREF _Toc512434571 h 54FUTURE SCOPE PAGEREF _Toc512434572 h 55REFERENCES PAGEREF _Toc512434573 h 56
List of Figures
TOC h z c “Figure” Figure 1 Inverse kinematics PAGEREF _Toc512433264 h 7Figure 2 centre of gravity PAGEREF _Toc512433265 h 8Figure 3 Flowchart for forward and backward movement without sensors PAGEREF _Toc512433266 h 11Figure 4 Graph of legs movement pattern PAGEREF _Toc512433267 h 12Figure 5 Flowchart for forward, backward, right and left movement with sensors PAGEREF _Toc512433268 h 15Figure 6 Walking pattern of leg for 1/4th cycle for one step movement PAGEREF _Toc512433269 h 16Figure 7 Side view of the robot PAGEREF _Toc512433270 h 17Figure 8 Top view of the robot PAGEREF _Toc512433271 h 17Figure 9 Hardware component used in robot – 1 PAGEREF _Toc512433272 h 20Figure 10 Hardware components used in robot-2 PAGEREF _Toc512433273 h 21Figure 11 PCB layout for power supply circuit PAGEREF _Toc512433274 h 28
center18535400
ACKNOWLEDGEMENT XE “Acknowledgement”
It would be impossible to name each and every individual who had offered the support encouragement and ideas which made this User Defined Project a possible thing.
We are greatly thankful to our external guide Prof. R. D. BHAGIYA of “Government Engineering College-Bhuj” who has given us a tremendous effort for shaping our ideas and knowledge according to the industrial requirements. He has helped us on reaching each and every milestone of the project, he has encouraged us in every failure during the project, and he has increased our confidence with every success in the project.
On the other hand, we are greatly thankful to our internal guide Prof. R. D. BHAGIYA who has made us aware with the requirement of the industry from this project, what we should actually try to learn that idea was planted by them and we had grown it up during this period with their helps.
We are also thankful to our H.O.D. DR. B. N. SUTHAR and other professors of Electrical department of our college.
Name of the Students
Chaudhari Pinakinkumar R. (140150109008)
Gajjar Darsh A. (140150109030)
Gamit Pragneshkumar J. (140150109032)
Kubavat Yash B. (140150109052)
Nikunjkumar Chaudhari (140150109061)
GTU COMPLETION CERITIFICATE
PLAGIARISM REPORT
center0ABSTRACT00ABSTRACT
Surveillance robot walking like an animal and it can remotely as well as automatically controllable (a bio robot which walking like an animal). A robot that can detect vicinity object and makes its path for walking in irregular and uneven surfaces, using servo motors on each leg make that possible. we are going to use control strategy for servo motors and object detective sensor. This robot is also used to where humans can’t easily reach at critical environment. A robot that can be used to analyse the local areas, surveillance at glaciers for research purpose and any radioactive areas and any hazardous place where any humans are not supposed to be there. With this project we want to contribute in society to do work by machine.
Chapter 1: INTRODUCTION-763471157215
Aim and ObjectiveAim of our project is to design and build a robot walking like an animal and it can remotely as well as automatically controllable (a bio robot which walking like an animal). A robot that can detect vicinity object and makes its path for walking in irregular and uneven surfaces, using servo motors on each leg make that possible. we are going to use control strategy for servo motors and object detective sensor. This robot is also used to where humans can’t easily reach at critical environment. A robot that can be used to analyse the local areas, surveillance at glaciers for research purpose and any radioactive areas and any hazardous place where any humans are not supposed to be there. With this project we want to contribute in society to do work by machine.
Objective of this project is to achieve our aim by using minimum resources to make walking robot with surveillance system. By using high torque metal gear servo motors in every joint of legs for walking at any irregular surfaces. Arduino Leonardo is used as the main brain of the system for dynamic walking gait of the robot. Arduino Uno is used for identifies the different environmental conditions by interfacing with different sensors. The live streaming of the video is done by using the raspberry pi and Picamera. The system has also features of object recognition and object tracking for the surveillance purpose. The body of the robot is made out of aluminium and acrylic because of its light weight, rigid construction and also its high weight carrying capacity. The dimension of the robot is 35cm*18cm*28cm.
Introduction1.2.1 Surveillance SystemTo observe particular places for changes around that place or for abnormal activity remotely from anywhere the surveillance system is very helpful. If you have to monitor your store you better to do have security cameras because, no one can afford to pay a human to do that job so in case of theft there is a prof of theft that who tried to theft etc. the exact imagery will be stored in the storage device while if you have employee to do that there might be problem in to recognizing the who is thief.
To monitor or watch the whole store a lot of human effort is required, and you have to pay more to the person who is allotted for that work. So, it is better to install CCTV system which is a surveillance system which keeps record of all the stuff in for of video footage. There is only installation cost and letter on you just have to pay for its maintenance which is very low compared to pay for a person who look after your store.
This system consists of high definition video cameras with night vision and high capacity storage to keep records for months or for years. And have more reliability and capable to operate in any kind of weather. And features like activation when motion, Remote monitoring capacity etc.
1.2.2 Introduction to robotsRobot is the device which operates or complete the given task as per the instruction given by the human or program. That task might be anything like task to drive around, to make coffee, welding, cutting, get the newspaper for you. The task can be anything robot can do any work which human thinks about expect few things. Robot is used everywhere like in industry, military, household, office etc.
There are different types of robots according to their operation and operating conditions. Some of robots are listed as below:
Types of Robots
According to nature –
? Spherical Robot – Spherical Robot is robot which has two rotating joints and one joint has liner motion.
? Cartesian Robot – Cartesian robot which operates in 3 liner axes.
? Articulated Robot – Articulated robot is a kind of leg of arm having 2 or more rotatory joints.
? SCARA Robot – SCARA robot is combination of spherical robot and Articulated robot.
? Delta Robot – Delta robot has pyramid like construction where pointy end is used as end part.
? Flying Robot – Flying Robot is a robot which has ability to fly like nowadays Drones are the one kind of flying robot (areal robot).
? Wheeled Robot – Wheeled Robot a robot which has ability to move around using wheels Rover is one of theme.
? Humanoid Robot – Humanoid Robot is robot which look like a human.
According to their place of operation
? Mobile robot – Mobile Robot is robot which can move around with the help of wheels or robot.
? Industrial robot – Industrial robot these robots are stationary and designed to perform the programmed task repeatedly. Spherical Robot, Cartesian Robot, Articulated Robot and SCARA robot are used as industry robots.
? Service robot – Service Robots are the robots who assists human or complete the dirty job like cleaning of room etc.
? Education robot – Educational robots are the robots who assists or help teacher in educational purposes.
? Modular robot – Modular robots are the robots which as ability gain or lose extra functionality by attaching or detaching part of robot.
? Collaborative robot – Collaborative robots are the robots which no harm to human anyhow.
In Military robots are used to defuse the bomb so the human life is saved by using the robot instead to do personally. Those robots are equipped with robotic hand to manipulate objects, Camera for video footage, Video transmitting unit for live video monitoring from remote places, various sensors to detect the metallic parts, to detect gases, fire sensors, Ultrasonic sensors for object detection, GPS for position, pressure sensor etc. For various purposes. Most of these kinds of robot have belt traction for manoeuvring around. With high capacity battery on it so it can operate complete wirelessly. So due to wheeled or belt traction it’s difficult to climb over a stair or some object’s like rock and bricks etc. To overcome the have special designed the traction system for the same. Which result in slow operation in that kind of situation. Even some robots are specially designed to do the same. But, due to that design they may have to compromise with its some of the features like low battery, reduced range, few functions, less payload capacity.
1.2.3 Surveillance RobotThe mobile form of the surveillance system which is combination of mobile robot and surveillance system is exist in various forms nowadays. So, what should be the function of Surveillance Robot?
? The Robot should be controlled by a wired or Wirelessly.
? The Robot should have ability to detect the object.
? The Robot should be able to avoid obstacle.
? The Robot Should be able to record the video.
? The Robot Should Run on its power for sufficient time.
? The Robot Should able to resolve its path.
? The robot Should able transmit data to observer.
? The Robot Should keep the records.
? The Robot can be controlled manually or automatically.
? The Robot Should able to recognize the surroundings.
Nowadays every place requires surveillance system to monitor what’s going on! Some of the areas where the remote observation is required but, it is not practically possible to install there a surveillance system paces like, War zones, Radioactive Zone, Forest, or the places where we want temporary surveillance is required. Or even to sniffing purposes we need the remote surveillance system. So, nowadays Surveillance robots are very popular like, Aerial Drone, Robots having cameras etc. This Surveillance Robot is designed with ability to walk on its 4 legs.
1.2.4 Why 4-legged Robot is Selected?Like with regulate wheeled or belt traction robot they seem to be look like vehicle so it looks odd in surrounding and it has some limitation of region of operation. Multi-tertian Robots which are able to work in different surfaces. But those robots compromise their other features. And due to wheeled and belt traction its affects the surface on which it is been used it drops its marks which is identical and disturb the surrounding environment. The payload capacity of the multi-tertian robots is less compared to other robots. And surveillance robot can be used for relief works like in rescuing if possible. If 4-legged surveillance robot is introduced to overcome the above-mentioned problems, it has better payload capacity, can be equipped with extra gadgets, even it doesn’t disturb the environment. Due to its animal like look it can be used for the filmography of forest life, like capturing films remotely of any herd of animals without disturbing their routine. Can be used for rescue purposes because it can carry extra stuff on it and has four leg which gives advantage in rough surface. Can be used for surveillance abandoned places without being noticed. It can be used like a pet dog which goes and sniff the stuff behind the wall! By adding the different environmental sensors, it able to detect the presence of hazardous gases or other matter or can detect fire.
Chapter 2: LITERATURE SURVEY-763471157215
For the designing of the robot, firstly develop the mathematical model of the robot. In a mathematical robot various parts to be considered i.e. links and joint. For the purpose of controlling the robot requires the position of each links and joint. So that for mathematical modelling is required to refer all geometrical and time-based properties of the motion. Solution of the mathematical modelling is given by theory of kinematics. Kinematic model describes the spatial position of the joints and links. The derivatives of kinematics deal with the mechanics of motion without considering the forces that cause it. The relationship between the motion and the forces and/or torques that cause them is due the dynamics problem.
Kinematics theory are as follows:
Inverse kinematics
Forward kinematics
Inverse KinematicsInverse kinematics used kinematics equations to find the angles of each joint to reach end-factor to one point (position) to another point linearly. this robot 2 degree of freedom in each leg so it is sufficient to consider the 2d system. In this robot the leg movement is controlled and loop is developed by using the inverse kinematics equation. The inverse kinematics equation of system is found by simply forming the triangles from the given joint points and the end-factor’s position and applying trigonometric equations. For the two joints it’s very easy to find the equation of inverse kinematics which is as follows.
Figure SEQ Figure * ARABIC 1 Inverse kinematics
As shown in above figure there is two joints which has angle of theta1 for 1st joint and for 2nd joint theta2 is the angle in inverse kinematics the angle of joints is the function of end factors position. If we consider that the first joint is placed at the origin point means at (0,0) point and end-factor position is (x, y). Then the equation of the theta1 and theta2 is given by as follows,
Theta2 = (a)cos(((m*m) + (n*n) – (y*y))/(2*m*n));
Theta1 = (a)cos(((m*m) – (n*n) + (y*))/(2*m*y)) – (a)tan(x1/y1);
In this method the angles of each joints are the function of the end-factors position. If we have to model the walking pattern of leg then have to consider the inverse kinematics. In inverse kinematics the trajectory is defined like may be linear or circular or parabolic as per the given function of end factor position the trajectory is defined.
2.2 Forward KinematicsForward kinematics is used to find the end-factor position by considering the joints angle. Which means that the end factor position is decides by the angles of the joints. That means the end factors position is the function of the angles of the joints.
The movement of arms is the Forward kinematics. In which end-factor’s trajectory is defined to reach to one position to another position as the function of angle of each joint.
2.3 Centre of GravityCentre of gravity in this kind of robot is very important because there are only 3 legs are on the ground to balance the body in steady place. Maintaining the centre of gravity in its position is important.
Figure SEQ Figure * ARABIC 2 centre of gravityAs shown in above figure in image 1 the cater of gravity is in its safe area and in second one the centre of gravity is out of its safe area.
Chapter 3: USER DEFINED PROBLEM FORMULATION ; DESIGN METHHOLOGY-763471157215
In design methodology of quadruped surveillance robot there are various steps to be taken are as below:
3.1 What is the practical function of the design?
? Movement
The coordinated movement control of multilegged robot has been a difficult problem in the field of robot because a robot needs to make a quick response to the change of external environment and various stimulus. The control strategy based on biological induction is a new control idea that has been gradually carried out in the multilegged robot researches, in which the pattern of movement of each leg of the quadruped robot is repeated in such a manner to move it forward or backward.
? Manipulation
To walk like a human or a cat, however, it is necessary to have more than one leg moving simultaneously in a pattern. Because one leg will move and other leg’s moving for making its centre of gravity balanced condition.
? Energy
This robot has several options for taking power for operation like we can connect 3*1.5 V cell to group of servo motor or by using 12V/9V adapter and dropdown its voltage using IC7805 and feed it to servo motor as well as Arduino for PWM generation or you can use any external power circuit where output voltage is 4.8-5.0 V for servo assembly and Arduino controller.
? Intelligence
To provide its intelligence we use it here a microcontroller named Arduino Leonardo that has capability of 7 PWM outputs and 12 Analog inputs and it can take decision according to how it is programmed to do particular operation.
? Sensing
Our robot works in various difficult environment that is not suitable for humans so its sensing elements must be capable to withstand that much of temperature, humidity, dirt muds, non-homogeneous surfaces, etc.
3.2 WalkingIt is for the most part a general approach that could be applied to other legged robots with multiple controllable joints in each leg for make that possible.
3.2.1 BasicsEach of the four legs has two joints known as the knee and the hip. The hip is a shoulder joint that moves the entire leg around an axis that runs vertically from forward to backward. The knee joint covers lower half part of leg and move it in also around vertical axis. Hence with two joints one gait movement possible.
We attached a servo motor for each joint. We have not thoroughly experimented with any settings aside from the recommended ones and use only the recommended ones for everything that is reported here.
The problem of compelling the robot to walk is greatly simplified by a technique called inverse kinematics. This technique allows the trajectory of a leg to be specified in terms of a three-dimensional trajectory for the foot. The inverse kinematics converts the location of the foot into the corresponding settings for the two joint angles. A precursor to deriving inverse kinematics formulas is having a model of the forward kinematics, the function that takes the two joint angles to a three-dimensional foot position. There is our mathematical model of the leg shown below.
Various attempts have been made to solve the problem of artificial automatic walking. These attempts include one-, two-, three-, four-, six- and eight-legged vehicles. Some principles of two-legged walking will be discussed later. When such machines are designed, the following problems must be solved:
Kinematics of the legs;
Control system providing the required sequence of leg movements;
Control of mechanical stability, especially for movement along a broken surface or an inclined plane.
Walking Flowchart & AlgorithmFollowing is the Flowchart for basic Forward and Backward movement without sensors
Figure SEQ Figure * ARABIC 3 Flowchart for forward and backward movement without sensorsAlgorithm
The basic forward motion algorithm is as follows which is implemented in following leg movement pattern which is shown by the graph.
Figure SEQ Figure * ARABIC 4 Graph of legs movement patternAs show above the three legs out of four legs moves in either direction each leg is 1/3 position ahead or 2/3 behind other reaming two legs. The movement in either direction is complete in total 3 section in which each leg is completes 1/3 portion out of 3 in each cycle while remaining 4th leg is restoring its position in air. the leg out of 3 which completes the 3 section of movement starts the restoring position in air and then that 4th leg from previous loop stats to complete the movement and this loop repeats for each leg for movement in either direction.
For programming Arduino, the following function is maid for the angle of joints
void motion (float x_1, float y_1, float* at, float* au)
{
float x1 = x_1;
//float y1=(x1/2) + 80;
float y1=y_1;
float c = sqrt((x1*x1) + (y1*y1));
float a2 = degrees((a)cos(((a*a) + (b*b) – (c*c))/(2*a*b)));
float a1 = degrees((a)cos(((a*a) – (b*b) + (c*c))/(2*a*c)) – (a)tan(x1/y1));
*at = 90 + a1;
*au = a2;
return;
}
Movement of each leg is controlled by following loop
for (float x= 0; x <= 52; x++)
{
if (x <= 13 && x >=0)
{
float x_1 = – x;
float y_1 = – 100;
motion (x_1, y_1, &theta1, &theta2);
s1.write(theta1);
s2.write(180-theta2);
float x_2 = -13 – x;
float y_2 = – 100;
}
}
Above loop is shown for 1/3 movement of one leg this type of loop is designed of each leg.
Our walking algorithm is based on this angular symmetry equation that based on inverse kinematics formulas are applied to achieve walking pattern.
For walking in forward direction robot needs to do a motion in x direction by taking constant height of C from ground to body and angle varies accordingly to it and movement of gait takes place.
When robot moves from position 1 as shown in figure and make some another position then tangent angle is used in equation to make that angle value perfectly vary over that linear direction.
Programming of motors do in such a manner to provide gait pattern while two or three legs move simultaneously in each time domain axis.
We have to required program and calibrate each servo motor for its perfect angle movement so if its position is slightly cross-aligned shaft with mounting axis then that error should must be minimized using calibration.
Figure SEQ Figure * ARABIC 5 Flowchart for forward, backward, right and left movement with sensors Above is the Flowchart for Forward, Backward, Right and Left movement with sensors
Walking pattern
Figure SEQ Figure * ARABIC 6 Walking pattern of leg for 1/4th cycle for one step movement
Design of Shape
Figure SEQ Figure * ARABIC 7 Side view of the robot
Figure SEQ Figure * ARABIC 8 Top view of the robotFor giving it shape like animal of four legged we must have to compromise with its stability during its operation hence for gait movement procedure required its body design similar like shown in figure. We can get rough idea about its basic shape from it.
In this shape, its centre of gravity depends how robot gait pattern is used because when each time two motor of each leg runs then COG distributed outer side of control body region makes it unbalancing and malfunctioning. So, for balance gait movement centre of gravity is major part hence shape of body is required in such a manner to make robot gait movement possible without fall on the floor.
We achieved this movement only in forward and reverse direction by using only two servos in each leg hence to make movement right and left we need external servo on hips to change its movement in horizontal axis. It will be added in future version.
Another main function of shape of body is to provide stiffness and robustness support of each part for moving safely after accidents means damage support.
What are the likely social and environmental effects of the design?
There is a need for vehicles that can travel in difficult terrain, where existing vehicles cannot go. Wheels excel on prepared surfaces such as rails and roads but perform poorly where the terrain is soft or uneven. Because of these limitations, only about half the earth’s landmass is accessible to existing wheeled and tracked vehicles, whereas a much greater area can be reached by animals on foot. It should be possible to build legged vehicles that can go to the places that animals can now reach.
From a different perspective, the small, discrete footholds of a legged machine will likely have a less significant impact on the environment than the continuous paths formed by wheeled and tracked machines. This means that the environment will also exert less drag on the legged machine. The legs can be raised out of soft soil in a direction normal to the terrain to minimize the energy losses associated with ploughing of the soil, thereby improving efficiency. The legs can also function as an active suspension system to provide a smoother ride over uneven terrain than would be experienced in a wheeled or tracked vehicle.
A number of engineering applications could benefit from the advantages legged machines offer in traversing rough terrain. Industrial uses include forestry and hazardous waste inspection and disposal, while research applications include the exploration of inaccessible or dangerous locations such as volcanos and the surfaces of other planets. Surveillance, reconnaissance, and land mine detection and removal are just a few examples of the many military use for legged machines.
Video Streaming and Object Recognition/TrackingFor surveillance purpose we use a raspberry pi and webcam combo for video streaming. First of all, we have to enable remote monitoring of field area through raspberry pi to any remote device by video streaming.
Secondly, we have to create a local server in raspberry pi and connect user into that network to enable them to communicate with our processor. After succeed in local server using “mjpg-server” we have to do image processing of fetched image frame, but another barrier is raspberry pi have 512 MB RAM and image processing need at-least one GB of processing RAM so we have to do this process into another high speed processor like another system that fetch image and process to detect image and recognize it using open source available libraries like “OpenCV”, “Imutils” and Python scientific libraries.
After setting up supported libraries into another system like fedora Linux operating system based on RedHat systems and host is loaded with Raspbian stretch operating system based on Debian. So, we setup VNC server into raspberry pi to enable GUI interfacing, and also use SSH.
Then after we succeed into remote monitoring then we have to use fire detection through it so we tried to solve that problem using OpenCV function so we first capture video and then convert into RGB to HSV, after that we did masking operation to feature extraction from that stream. And then using gaussian selection function we add a counter lines around selected range of colour and make mask.
Another feature we add by image processing is to develop an object recognition or in another word is to develop small sense or intelligence to recognize different object and give name according to their type. So, we fetch fetched some details from trained models of different image classifiers and then make blobs then use it object feature matching with data from streaming. It also shows confidence level of recognized object and give it name from prototype text. So, this whole process is done through “Imutils” library contains all these features to use open source into any selected region.
Chapter 4: USER DEFINED PROBLEM SOLUTION/ HARDWARE COMPONENTS DESCRIPTION-77415637102
Figure SEQ Figure * ARABIC 9 Hardware component used in robot – 1
Figure SEQ Figure * ARABIC 10 Hardware components used in robot-2
Raspberry Pi Zero W
1GHz, Single-core CPU
512MB RAM
Mini HDMI and USB On-The-Go ports
Micro USB power
HAT-compatible 40-pin header
Composite video and reset headers
CSI camera connector
802.11n wireless LAN
Bluetooth 4.0
Arduino Leonardo
Microcontroller ATmega32u4
Operating Voltage 5V
Input Voltage (Recommended) 7-12V
Input Voltage (limits) 6-20V
Digital I/O Pins 20
PWM Channels 7
Analog Input Channels 12
DC Current per I/O Pin 40 mA
DC Current for 3.3V Pin 50 mA
Flash Memory 32 KB (ATmega32u4) of which 4 KB used by bootloader
SRAM 2.5 KB (ATmega32u4)
EEPROM 1 KB (ATmega32u4)
Clock Speed 16 MHz
Length 68.6 mm
Width 53.3 mm
Weight 20 g
Adafruit 16-channel 12-bit PWM Servo Driver – I2C Interface – PCA9685
General Specification:
Adjustable frequency PWM up to about 1.6 KHz
12-bit resolution for each output – for servos, that means about 4us resolution at 60Hz update rate
Configurable push-pull or open-drain output
Output enable pin to quickly disable all the outputs
Terminal block for power input
Reverse polarity protection on the terminal block input
Green power-good LED
3 pin connectors in groups of 4 so you can plug in 16 servos at once
Chainable design
A spot to place a big capacitor on the V+ line
220-ohm series resistors on all the output lines to protect them, and to make driving LEDs trivial
Technical Specification:
Dimensions (no headers or terminal block) 2.5″ x 1″ x 0.1″ (62.5mm x 25.4mm x 3mm)
Weight (no headers or terminal block): 5.5grams
Weight (with 3×4 headers & terminal block): 9grams
This board/chip uses I2C 7-bit address between 0x40-0x7F, selectable with jumpers
Pi camera Module
Hardware specification:
Camera Module v1
Net price $25
Size Around 25 × 24 × 9 mm
Weight 3g
Still resolution 5 Megapixels
Video modes 1080p30, 720p60 and 640 × 480p60/90
Linux integration V4L2 driver available
C programming API OpenMAX IL and others available
Sensor OmniVision OV5647
Sensor resolution 2592 × 1944 pixels
Sensor image area 3.76 × 2.74 mm
Pixel size 1.4 µm × 1.4 µm
Optical size 1/4″
Full-frame SLR lens equivalent 35 mm
S/N ratio 36 dB
Dynamic range 67 dB @ 8x gain
Sensitivity 680 mV/lux-sec
Dark current 16 mV/sec @ 60 C
Well capacity 4.3 Ke-
Fixed focus 1 m to infinity
Focal length 3.60 mm +/- 0.01
Horizontal field of view 53.50 +/- 0.13 degrees
Vertical field of view 41.41 +/- 0.11 degrees
Focal ratio (F-Stop) 2.9
Software Features
Picture formats JPEG (accelerated), JPEG + RAW, GIF, BMP, PNG, YUV420, RGB888
Video formats raw h.264 (accelerated)
Effects negative, solarise, posterize, whiteboard, blackboard, sketch, denoise, emboss, oilpaint, hatch, gpen, pastel, watercolour, film, blur, saturation
Exposure modes auto, night, nightpreview, backlight, spotlight, sports, snow, beach, verylong, fixedfps, antishake, fireworks
Metering modes average, spot, backlit, matrix
Automatic white balance modes off, auto, sun, cloud, shade, tungsten, fluorescent, incandescent, flash, horizon
Triggers Keypress, UNIX signal, timeout
Extra modes demo, burst/timelapse, circular buffer, video with motion vectors, segmented video, live preview on 3D models
Ultrasonic Sensor
HC-SR04: It has 4 pins which are VCC(power), Trig(Trigger), Echo(Receive), and GND(Ground). It provides range of 2cm to 400cm.
MG996R Servo
Specifications:
Weight: 55 g
Dimension: 40.7 x 19.7 x 42.9 mm approx.
Stall torque: 9.4 kgf·cm (4.8 V), 11 kgf·cm (6 V)
Operating speed: 0.17 s/60º (4.8 V), 0.14 s/60º (6 V)
Operating voltage: 4.8 V a 7.2 V
Running Current 500 mA – 900 mA (6V)
Stall Current 2.5 A (6V)
Dead band width: 5 µs
Stable and shock proof double ball bearing design
Temperature range: 0 ºC – 55 ºC
16×2 LCD Display
General Features
Display Mode: STN, BLUB
Display Format: 16 Character x 2 Line
Viewing Direction: 6 O’clock
Input Data: 4-Bits or 8-Bits interface available
Display Font: 5 x 8 Dots
Power Supply: Single Power Supply (5V±10%)
Driving Scheme: 1/16Duty,1/5Bias
BACKLIGHT?SIDE??LED?WHITE?
Pin Description
Pin No Function Name
1 Ground (0V) Ground
2 Supply voltage; 5V (4.7V – 5.3V) Vcc
3 Contrast adjustment; through a variable resistor VEE
4 Selects command register when low; and data register when high Register Select
5 Low to write to the register; High to read from the register Read/write
6 Sends data to data pins when a high to low pulse is given Enable
7 8-bit data pins DB0
8 DB1
9 DB2
10 DB3
11 DB4
12 DB5
13 DB6
14 DB7
15 Backlight VCC (5V) Led+
16 Backlight Ground (0V) Led-
LM35 (Precision Centigrade Temperature Sensors)
Calibrated Directly in Celsius (Centigrade)
Linear + 10-mV/°C Scale Factor
0.5°C Ensured Accuracy (at 25°C)
Rated for Full ?55°C to 150°C Range
Suitable for Remote Applications
Low-Cost Due to Wafer-Level Trimming
Operates From 4 V to 30 V
Less Than 60-?A Current Drain
Low Self-Heating, 0.08°C in Still Air
Non-Linearity Only ±¼°C Typical
Low-Impedance Output, 0.1 ? for 1-mA Load
Light Dependent Resistor (LDR)
Specifications
Arduino Uno
Max power dissipation 200mW
Max voltage @ 0 lux 200V
Peak wavelength 600nm
Min. resistance @ 10lux 1.8k?
Max. resistance @ 10lux 4.5k?
Typ. resistance @ 100lux 0.7k?
Dark resistance after 1 sec 0.03M?
Dark resistance after 5 secs 0.25M?
Microcontroller ATmega328
Operating Voltage 5V
Input Voltage (Recommended) 7-12V
Input Voltage (limits) 6-20V
Digital I/O Pins 14
PWM Channels 6
Analog Input Channels 6
DC Current per I/O Pin 40 mA
DC Current for 3.3V Pin 50 mA
Flash Memory 32 KB (ATmega328) of which
0.5 KB used by bootloader
SRAM 2 KB (ATmega328)
EEPROM 1 KB (ATmega328)
Clock Speed 16 MHz
Length 68.6 mm
Width 53.3 mm
Weight 20 g
Power supply
Figure SEQ Figure * ARABIC 11 PCB layout for power supply circuitSpecifications:
Input voltage: 12V
Output voltage: 6.8V
Resistors: 60ohm(2W), 10K ohm(1W), 2K2 ohm(2W)
Capacitor: 1000µF(50V), 470µF(63V), 100µF(25V)
Transistors: 2N3055(NPN, 2nos)
Zener diode: 7V
Max. Output Current: 15Amp
Chapter 5: HARDWARE IMPLEMENTATION/ SOFTWARE DETAILS WITH PROGRAM-763471157215
Arduino Programming for Dynamic Walking Gait
//including required library
#include <PinChangeInterrupt.h>
#include <Wire.h>
#include <Adafruit_PWMServoDriver.h>
#include <NewPing.h>
Adafruit_PWMServoDriver pwm = Adafruit_PWMServoDriver();
const byte channel_pin = {8, 9, 10};
volatile unsigned long rising_start = {0, 0, 0};
volatile long channel_length = {0, 0, 0};
#define TRIGGER_PIN 7
#define ECHO_PIN 6
#define MAX_DISTANCE 600
int thresh = 20;
long randNumber;
//Defining upper bone size and lower bone size of leg
NewPing sonar(TRIGGER_PIN, ECHO_PIN, MAX_DISTANCE);
#define a 75
#define b 100
#define h -165
//Defining angles of each joints
float theta1,theta2,theta3,theta4,theta5,theta6,theta7,theta8;
float t1,t2,t3,t4,t5,t6,t7,t8;
//Defining Left and right leg max move limit to lean left or right
float r,l;
float x9;
float y9;
const int ps = {2,3,4,5};
int dy5;
int ytemp = 0;
void setup()
{
//initializing baudrate and frequency of PCA9685
Serial.begin(9600);
pinMode(channel_pin0, INPUT);
pinMode(channel_pin1, INPUT);
pinMode(channel_pin2, INPUT);
attachPinChangeInterrupt(digitalPinToPinChangeInterrupt(channel_pin0), onRising0, CHANGE);
attachPinChangeInterrupt(digitalPinToPinChangeInterrupt(channel_pin1), onRising1, CHANGE);
attachPinChangeInterrupt(digitalPinToPinChangeInterrupt(channel_pin2), onRising2, CHANGE);
pwm.begin();
pwm.setPWMFreq(60);
delay(10);
for(int i = 0 ; i<=3;i++)
{
pinMode(psi,INPUT);
}
}
//Following function is IK function which defines the angle of joints according to x and y cordinate
void processPin(byte pin) {
uint8_t trigger = getPinChangeInterruptTrigger(digitalPinToPCINT(channel_pinpin));
if(trigger == RISING) {
rising_startpin = micros();
} else if(trigger == FALLING) {
channel_lengthpin = micros() – rising_startpin;
}
}
void onRising0(void) {
processPin(0);
}
void onRising1(void) {
processPin(1);
}
void onRising2(void) {
processPin(2);
}
void motion(float x_1,float y_1,float* at,float* au)
{ float x1 = x_1;
float y1=y_1;
float c = sqrt((x1*x1)+(y1*y1));
float a2 = degrees(acos(((a*a)+(b*b)-(c*c))/(2*a*b)));
float a1 = degrees(acos(((a*a)-(b*b)+(c*c))/(2*a*c))+atan(x1/y1));
*at = 90 + a1;
*au = a2;
return;
}
//following Function to program leg
void leg(int i,int lg,int minv,int maxv,int cmin,int cmax,float k,int fly,int um,int unm,int lm,int lnm,int p1,int p2,int hg)
{
if ( fly == 0)
{
xlg = map(i,minv,maxv,cmin*k,cmax*k);
ylg = hg;
motion(xlg,ylg,&theta1,&theta2);
int t1 = map(theta1,0,180,um,unm);
int t2 = map(theta2 – 90 ,0,180,lm,lnm);
pwm.setPWM(p1,0,t1);
pwm.setPWM(p2,0,t2);
// Serial.print(x1);
// Serial.print(” | “);
}
if ( fly == 1)
{
xlg = map(i,minv,maxv,cmin*k,cmax*k);
ylg = hg+25;
//ylg = -(((hg+155)*xlg*xlg)/(1600*k*k))-155;
motion(xlg,ylg,&theta1,&theta2);
int t1 = map(theta1,0,180,um,unm);
int t2 = map(theta2 – 90 ,0,180,lm,lnm);
pwm.setPWM(p1,0,t1);
pwm.setPWM(p2,0,t2);
// Serial.print(x1);
// Serial.print(” | “);
// Serial.println(“”);
}
if ( fly == 2)
{
xlg = hg;
ylg = map(i,minv,maxv,cmin,cmax);;
//ylg = -(((h+155)*xlg*xlg)/(1600*k*k))-155;
motion(xlg,ylg,&theta1,&theta2);
int t1 = map(theta1,0,180,um,unm);
int t2 = map(theta2 – 90 ,0,180,lm,lnm);
pwm.setPWM(p1,0,t1);
pwm.setPWM(p2,0,t2);
//Serial.print(x1);
// Serial.print(” | “);
// Serial.println(“”);
}
}
void rotation(int x,int d,int* dr)
{
if(d == 0)
{
*dr = map(x,0,240,0,240);
}
if(d == 1)
{
*dr = map(x,0,240,240,0);
}
}
//following function for n steps
void cw(int lp,int dir,float turnleft,float turnright,int hig)
{
for ( int j = 0; j <= lp; j++)
{
for ( int dx = 0; dx <= 240; dx++)
{
int x;
rotation(dx,dir,&x);
// Serial.print(x);
// Serial.print(” | “);
//Serial.println(“”);
// usage of leg(x,leg,minval of x,max val of x,min x of leg,max x of leg ,0 to 1 to reduce x of leg ,curve or line,uppper servo mlim,uppper servo mxlim,lower servo mlim,lower servo mlim,0,1);
if (x<30 && x>=0)
{
//these legs moves on ground
leg(x,1,0,30,40,20,turnleft,0,170,550,500,180,0,1,hig);//leg1
leg(x,2,0,30,0,-20,turnright,0,520,170,165,555,2,3,hig);//leg2
leg(x,3,0,30,-20,0,turnleft,0,500,155,160,540,4,5,hig);//leg3
leg(x,4,0,30,20,40,turnright,0,170,525,525,170,6,7,hig);//leg4
}
if (x<40 && x>=30)
{
leg(x,1,30,40,-155,-145,turnleft,2,170,550,500,180,0,1,20);//leg1
leg(x,2,30,40,-155,-165,turnright,2,520,170,165,555,2,3,-20);//leg2
}
if (x<50 && x>=40)
{
//this leg will lift up from ground and then moves
leg(x,4,40,50,40,-40,turnright,1,170,525,525,170,6,7,-155);//leg4
}
if (x<60 && x>=50)
{
leg(x,1,50,60,-145,-155,turnleft,2,170,550,500,180,0,1,20);//leg1
leg(x,2,50,60,-165,-155,turnright,2,520,170,165,555,2,3,-20);//leg2
}
if (x<90 && x>=60)
{
leg(x,1,60,90,20,0,turnleft,0,170,550,500,180,0,1,hig);//leg1
leg(x,2,60,90,-20,-40,turnright,0,520,170,165,555,2,3,hig);//leg2
leg(x,3,60,90,0,20,turnleft,0,500,155,160,540,4,5,hig);//leg3
leg(x,4,60,90,-40,-20,turnright,0,170,525,525,170,6,7,hig);//leg4
}
if (x<100 && x>=90)
{
leg(x,3,90,100,-155,-145,turnleft,2,500,155,160,540,4,5,20);//leg3
leg(x,4,90,100,-155,-165,turnright,2,170,525,525,170,6,7,-20);//leg4
}
if (x<110 && x>=100)
{
leg(x,2,100,110,-40,40,turnright,1,520,170,165,555,2,3,-155);//leg2
}
if (x<120 && x>=110)
{
leg(x,3,110,120,-145,-155,turnleft,2,500,155,160,540,4,5,20);//leg3
leg(x,4,110,120,-165,-155,turnright,2,170,525,525,170,6,7,-20);//leg4
}
if (x<150 && x>=120)
{
leg(x,1,120,150,0,-20,turnleft,0,170,550,500,180,0,1,hig);//leg1
leg(x,2,120,150,40,20,turnright,0,520,170,165,555,2,3,hig);//leg2
leg(x,3,120,150,20,40,turnleft,0,500,155,160,540,4,5,hig);//leg3
leg(x,4,120,150,-20,0,turnright,0,170,525,525,170,6,7,hig);//leg4
}
if (x<160 && x>=150)
{
leg(x,2,150,160,-155,-145,turnright,2,520,170,165,555,2,3,20);//leg2
leg(x,1,150,160,-155,-165,turnleft,2,170,550,500,180,0,1,-20);//leg1
}
if (x<170 && x>=160)
{
leg(x,3,160,170,40,-40,turnleft,1,500,155,160,540,4,5,-155);//leg3
}
if (x<180 && x>=170)
{
leg(x,2,170,180,-145,-155,turnright,2,520,170,165,555,2,3,20);//leg2
leg(x,1,170,180,-165,-155,turnleft,2,170,550,500,180,0,1,-20);//leg1
}
if (x<210 && x>=180)
{
leg(x,1,180,210,-20,-40,turnleft,0,170,550,500,180,0,1,hig);//leg1
leg(x,2,180,210,20,0,turnright,0,520,170,165,555,2,3,hig);//leg2
leg(x,3,180,210,-40,-20,turnleft,0,500,155,160,540,4,5,hig);//leg3
leg(x,4,180,210,0,20,turnright,0,170,525,525,170,6,7,hig);//leg4
}
if (x<220 && x>=210)
{
leg(x,4,210,220,-155,-145,turnright,2,170,525,525,170,6,7,20);//leg4
leg(x,3,210,220,-155,-165,turnleft,2,500,155,160,540,4,5,-20);//leg3
}
if (x<230 && x>=220)
{
leg(x,1,220,230,-40,40,turnleft,1,170,550,500,180,0,1,-155);//leg1
}
if (x<240 && x>=230)
{
leg(x,4,230,240,-145,-155,turnright,2,170,525,525,170,6,7,20);//leg4
leg(x,3,230,240,-165,-155,turnleft,2,500,155,160,540,4,5,-20);//leg3
}
delay(15);
}
}
}
void loop()
{
int ytemp=0;
//following function is used to move steps usage cw(no of step,forward and reverse using 0 and 1,to reduse left lege movement val between 0 to 1 ,same for right as left)
int uS = sonar.ping_cm();
Serial.println(uS);
int pwm = -155; //map(channel_length0,1024,1900,-165,-135);
//if (channel_length1>1600)
//{
if (uS <=25 && uS>0)
{
cw(1,0,1,.5,pwm);
}
else
{
cw(1,1,.5,.5,pwm);
}
// }
/* if (channel_length1<1300)
{
cw(1,0,1,1,pwm);
}*/
}
void dynamicY(int l,int x,int minR,int maxR,int tominR,int tomaxR,int* yd)
{
int k = digitalRead(psl);
if(k == LOW)
{
dyl = map(x,minR,maxR,tominR,tomaxR);
*yd = dyl;
ytemp = dyl;
return;
}
if(k == HIGH)
{
*yd = ytemp;
return;
}
}
Arduino Program ExplanationIn first portion of program required libraries are included Which are as follows.
#include <PinChangeInterrupt.h> This library is used to perform pin change interrupts which helps to trigger interrupt at multiple input pins when their input is available. Because Arduino has very limited interrupt enabled pins.
#include <Wire.h> this library is used to communicate with PCA9685 IC using SCL and SDA pins.
#include <Adafruit_PWMServoDriver.h> this library is used for PCA9685 operation.
#include <NewPing.h> This library is for Ultrasonic sensor feedback.
To know on which address the PCA9685 is connected following “Adafruit_PWMServoDriver pwm = Adafruit_PWMServoDriver (); ” command is used. Empty bracket means default address.
After that the leg size is defined using following commands.
#define a 75 lower part size in mm.
#define b 100 upper part size in mm.
#define h -165 default body base Hight from ground in mm.
To set up the arduino for operation setup code is placed in void setup. In void setup the baudrate at which arduino communicates with peripheral devices like PCA9685 rx tx revicer module, Bluetooth module etc. To Set baudrate “Serial.begin(9600);” is used.
To start pwm of PCA9685 “pwm.begin();” is used without this there is no pwm output. To Set the frequecy of PCA9685 IC “pwm.setPWMFreq(60);” is used where 60 is used for 60 Hz frequency. To aperate anlog servo pwm frequency is kept 50-60Hz. And for digital servo pwm frequency is 1000Hz.
void motion(float x_1,float y_1,float* at,float* au)
{ float x1 = x_1;
float y1=y_1;
float c = sqrt((x1*x1)+(y1*y1));
float a2 = degrees(acos(((a*a)+(b*b)-(c*c))/(2*a*b)));
float a1 = degrees(acos(((a*a)-(b*b)+(c*c))/(2*a*c))+atan(x1/y1));
*at = 90 + a1;
*au = a2;
return;
}
Above code is the function IK for two joints of leg. This function requires two input variable which are the 2d coordinates of end factor of leg. And returns the joint angle. Above code is mathematical modelling of inverse kinematics.
void leg(int i,int lg,int minv,int maxv,int cmin,int cmax,float k,int fly,int um,int unm,int lm,int lnm,int p1,int p2,int hg)
{
if ( fly == 0)
{
xlg = map(i,minv,maxv,cmin*k,cmax*k);
ylg = hg;
motion(xlg,ylg,&theta1,&theta2);
int t1 = map(theta1,0,180,um,unm);
int t2 = map(theta2 – 90 ,0,180,lm,lnm);
pwm.setPWM(p1,0,t1);
pwm.setPWM(p2,0,t2);
// Serial.print(x1);
// Serial.print(” | “);
}
if ( fly == 1)
{
xlg = map(i,minv,maxv,cmin*k,cmax*k);
ylg = hg+25;
//ylg = -(((hg+155)*xlg*xlg)/(1600*k*k))-155;
motion(xlg,ylg,&theta1,&theta2);
int t1 = map(theta1,0,180,um,unm);
int t2 = map(theta2 – 90 ,0,180,lm,lnm);
pwm.setPWM(p1,0,t1);
pwm.setPWM(p2,0,t2);
// Serial.print(x1);
// Serial.print(” | “);
// Serial.println(“”);
}
if ( fly == 2)
{
xlg = hg;
ylg = map(i,minv,maxv,cmin,cmax);;
//ylg = -(((h+155)*xlg*xlg)/(1600*k*k))-155;
motion(xlg,ylg,&theta1,&theta2);
int t1 = map(theta1,0,180,um,unm);
int t2 = map(theta2 – 90 ,0,180,lm,lnm);
pwm.setPWM(p1,0,t1);
pwm.setPWM(p2,0,t2);
//Serial.print(x1);
// Serial.print(” | “);
// Serial.println(“”);
}
}
This function is responsible for leg movement. This function requires 9 arguments.
void leg(int i,int lg,int minv,int maxv,int cmin,int cmax,float k,int fly,int um,int unm,int lm,int lnm,int p1,int p2,int hg)
Int i : time variable. Int lg : no of leg.Int minv , int maxv & int cmin, int cmax : to map ” i ”
For example, i is varying from 0 to 240 and we want to move legs end factor –40 to –20 (hight kept constant which is Y cordinate and –40 to –20 is X cordinate) when i changes 0 to 20. Then it canbe written as 0,20,-40,-20.
Float k is used to decrease the total movement of end factor by k fraction. Int fly is used to chose whether leg moves on ground or restores its position in air. Int um, int unm, int lm, int lnm is used to map servo angle between 0 to 180 degree. Int pi, int p2 are used to define the location of servo on PCA9685IC and int hg is used to set the Hight of base from ground.
void cw(int lp,int dir,float turnleft,float turnright,int hig)
Under this function for loop is placed which runs for 0 to 240. This loop can be run from 240 to 0 for reverse direction as well. This can be done by “rotation(dx,dir,&x);” this function retuns 0 to 240 if dir = 1 is taken and returns 240 to 0 is dir =0 is taken.
1/4th loop is gives as follows.
if (x<30 && x>=0)
{
//these legs moves on ground
leg(x,1,0,30,40,20,turnleft,0,170,550,500,180,0,1,hig);//leg1
leg(x,2,0,30,0,-20,turnright,0,520,170,165,555,2,3,hig);//leg2
leg(x,3,0,30,-20,0,turnleft,0,500,155,160,540,4,5,hig);//leg3
leg(x,4,0,30,20,40,turnright,0,170,525,525,170,6,7,hig);//leg4
}
if (x<40 && x>=30)
{
leg(x,1,30,40,-155,-145,turnleft,2,170,550,500,180,0,1,20);//leg1
leg(x,2,30,40,-155,-165,turnright,2,520,170,165,555,2,3,-20);//leg2
}
if (x<50 && x>=40)
{
//this leg will lift up from ground and then moves
leg(x,4,40,50,40,-40,turnright,1,170,525,525,170,6,7,-155);//leg4
}
if (x<60 && x>=50)
{
leg(x,1,50,60,-145,-155,turnleft,2,170,550,500,180,0,1,20);//leg1
leg(x,2,50,60,-165,-155,turnright,2,520,170,165,555,2,3,-20);//leg2
}
In above code when x is between 0 and 30 all leg will move keeping difference of ¼ step whit previous and next leg end factor position.eg. Leg 1 will move 40 to 20, leg 3 will move -20 to 0 (leg 3 is mirror image of leg 1 hence 20 to 0 is –20 to 0 for correct direction movement), Leg2 will move 0 to –20, Leg 4 will move 20 to 40 (leg 4 is mirror image of leg 2 hence –20 to –40 is 20 to 40).
After 0 to 30 intervals and in interval of 30 to 40 all legs end factors stays stand still and the height of leg 1 will decrease from –155 to –145(legs are in third and fourth quadrant hence height is taken minus). And leg 2 will increase Hight from –155 to –165. This will shift the Centre of gravity towards the leg 1 corner. This will help robot to stand on three legs while fourth leg is suspended in air.
After that in interval of 40 to 50 leg 4 will move 40 to –40 in air and restores its position. After that in interval of 50 to 60 leg 1 and leg 2 restores its Hight back to –155.
After that the second 1/4th cycle starts in which second leg will restore its position. Then in third 1/4th cycle leg 3 will restores its position. Then in last 1/4th cycle leg 1 will restores its position.
To execute all those prcess program is written in viod loop.
In void loop
void loop()
{
int ytemp=0;
//following function is used to move steps usage cw(no of step, forward and reverse using 0 and 1, to reduce left leg movement Val between 0 to 1 ,same for right as left)
int uS = sonar.ping_cm();
Serial.println(uS);
int pwm = -155; //map(channel_length0,1024,1900, -165, -135);
//if (channel_length1>1600)
//{
if (uS <=25 && uS>0)
{
cw(1,0,1,.5,pwm);
}
else
{
cw(1,1,.5,.5,pwm);
}
// }
/* if (channel_length1<1300)
{
cw(1,0,1,1,pwm);
}*/
}
When there is wall or any big object with in the 25cm range robot will move in reverse direction and turn. If there is nothing then then robot will move normally.
Object Recognition using Deep Neural Networks and OpenCV Library of Trained Models and Blobs to Image Conversionfrom imutils.video import VS
from imutils.video import FPS
import numpy as npi
import argparse
import imutils
import time
import cv2
ap = argparse.ArgumentParser()
ap.add_argument(“-p”, “–prototxt”, required=True,
help=”path to Caffe ‘deploy’ prototxt file”)
ap.add_argument(“-m”, “–model”, required=True,
help=”path to Caffe pre-trained model”)
ap.add_argument(“-c”, “–confidence”, type=float, default=0.2,
help=”minimum probability to filter weak detections”)
args = vars(ap.parse_args())
CLASSES = “background”, “aeroplane”, “bicycle”, “bird”, “boat”,
“bottle”, “bus”, “car”, “cat”, “chair”, “cow”, “table”,
“dog”, “horse”, “motorbike”, “person”, “plant”, “sheep”,
“sofa”, “train”, “mobile”
COLORS = npi.random.uniform(0, 255, size=(len(CLASSES), 3))
print(“model…is loaded”)
net = cv2.dnn.readNetFromCaffe(args”prototxt”, args”model”)
print(“video stream… is started”)
vs = VS(src=’http://192.168.43.111:9000/?action=stream’).start()
time.sleep(2.0)
fps = FPS().start()
while True:
frame = vs.read()
frame = imutils.resize(frame, width=400)
(h, w) = frame.shape:2
blobs = cv2.dnn.blobFromImage(cv2.resize(frame, (300, 300)),
0.007843, (300, 300), 127.5)
net.setInput(blobs)
detections = net.forward()
for i in npi.arange(0, detections.shape2):
confidence = detections0, 0, i, 2
if confidence > args”confidence”:
idx = int(detections0, 0, i, 1)
box = detections0, 0, i, 3:7 * npi.array(w, h, w, h)
(startX, startY, endX, endY) = box.astype(“int”)
label = “{}: {:.2f}%”.format(CLASSESidx,
confidence * 100)
cv2.rectangle(frame, (startX, startY), (endX, endY),
COLORSidx, 2)
y = startY – 15 if startY – 15 > 15 else startY + 15
cv2.putText(frame, label, (startX, y),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, COLORSidx, 2)
cv2.imshow(“Frame”, frame)
key = cv2.waitKey(1) & 0xFF
if key == ord(“a”):
break
fps.update()
fps.stop()
print(” elapsed time: {:.2f}”.format(fps.elapsed()))
print(” approx. FPS: {:.2f}”.format(fps.fps()))
# do a lil-bit of cleanup
cv2.destroyAllWindows()
vs.stop()
5.4 Object Detection and Mask Creation as per colour HSV value/ Python and Tkinter based GUI version of Video Stream Facilitates with Taking Screenshotimport numpy as np
import datetime as dt
import os
import cv2
import tkinter as tk
#arrange GUI
window = tk.Tk()
#Makes main window
window.wm_title(“Wolfy’s Eye”)
window.config(background=”#FFFFFF”)
#Graphics window
imageFrame = tk.Frame(window, width=600, height=500)
imageFrame.grid(row=0, column=0, padx=10, pady=2)
#Capture video frames
lmain = tk.Label(imageFrame)
lmain.grid(row=0, column=0)
cap = cv2.VideoCapture(0)
def frame2():
lowerBound = np.array(33, 80, 40)
upperBound = np.array(102, 255, 255)
kernelOpen = np.ones((5, 5))
kernelClose = np.ones((20, 20))
font = cv2.FONT_HERSHEY_SIMPLEX
while True:
_, img = cap.read()
img = cv2.resize(img, (340, 220))
img = cv2.flip(img, 1)
# convert BGR to HSV
imgHSV = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# creating the Mask
mask = cv2.inRange(imgHSV, lowerBound, upperBound)
# use of morphology function
maskOpen = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernelOpen)
maskClose = cv2.morphologyEx(maskOpen, cv2.MORPH_CLOSE, kernelClose)
maskFinal = maskClose
_, conts, h = cv2.findContours(maskFinal.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cv2.drawContours(img, conts, -1, (255, 0, 0), 3)
for i in range(len(conts)):
x, y, w, h = cv2.boundingRect(contsi)
cv2.rectangle(img, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.putText(img, str(i + 1), (x, y + h), font, 1, (0, 255, 255), 1)
cv2.imshow(“cam”, img)
cv2.waitKey(10)
def snap():
_, img = cap.read()
t = dt.datetime.now()
filename = “{}.jpg”.format(t.strftime(“%Y-%m-%d_%H-%M-%S”))
p = os.path.sep.join((‘/home/dgajjar/PycharmProjects/py3cv’, filename))
cv2.imwrite(p, img.copy())
print(“image saved {}”.format(filename))
#show_frame ()
sFrame2 = tk.Frame (window, width=600, height=100)
sFrame2.grid(row = 600, column=1, padx=50, pady=2)
butn = tk.Button(sFrame2, text=’screenshot’, command=snap()).grid(row=1, column=0)
frame2()
window.mainloop()
Chapter 6: SUMMARY OF RESULTS-763471157215
6.1 AdvantagesIt has high work on time compare to drones and other mammal type robots.
It can monitor and sense real time data from vicinity areas.
It can Passover through objects and also walks over uneven surface without any break.
It can measure distance of object aligned with range of ultrasonic sensor.
6.2 DisadvantagesIt need some manual input from programmer to start its required task.
It required regulated high ampere dc power supply.
It has need complex programming structure to develop artificial intelligence that required more memory for processing (because raspberry pi zero has only 512MB RAM).
Image processing also can’t have done at host side due to low memory so client computer required.
CONCLUSION-688406154940
After making this project we can conclude that this robot with less degree of freedom (2 DOF) it can perform basic manoeuvre easily. It can move forward and backward easily but turning left or right is bit difficult because of 2 degrees of freedom.
Object detection, colour detection and object discrimination are carried out by the raspberry pi zero and pi camera v1.3 and OpenCV. Video output is shown to anyone within the Local Area Network or Wide Area Network. Ultrasonic sensor is used to detect object within range of 25cm.
Robot is equipped with 2200mAH, 11.1 Volts Lithium-Polymer battery which is enough to run robot for 20mins to 30mins when battery is full charged. About 6.8 Amps current is sink by the robot at no-load. Voltage is regulated to 6.80 Volts and 5 Volts suing two different voltage regulating circuits. 5 Volts is for peripheral devices and 6.80 Volts for servos.
FUTURE SCOPE-664466165488
First of all, this project is still under development because aim behind this project is very broad, a surveillance robot contains till two main features that takes our focus like it has Walking gait same as cat, dog etc. and can recognize selected object and keep eyes on them as its programmed.
So, next step for development is to carry out some artificial intelligence techniques to its vision and control variables by which it can take decision by supervised learning from trained data. This is very challenging task but it is requirement of future development. This type of mammal robots cannot defeat drone on some particular area of interest but stability of operation and long-lasting monitoring of real time process on different zone that is hazardous for drone as well as living creatures also make a clear standing position for this type of robot.
In future we can use it for mine survey, as a rover, as a remote server, as a pet that helps humans in their day to day routine cumbersome work. Above mentioned task is fulfilled if we develop an intelligence database and programmes to some higher degree of accuracy using structuring neural networks and also use high speed processor that can-do multithreading easily and always stays connected through internet to sync its databases That’s it!
REFERENCESRobotics and Control by R K Mittal and I J Nagrath
Robot Programming a Guide to Controlling Autonomous Robots by Cameron Hughes and Tracey Hughes
http://www.instructables.com/id/QUATTRO-the-Arduino-QuadrupedRobothttp://www.benripley.com/diy/arduino/three-ways-to-read-a-pwm-signal-with-arduinohttps://howtomechatronics.com/tutorials/arduino/arduino-and-hc-05bluetooth-module-tutorialhttp://www.ti.com/lit/ds/symlink/lm35.pdfhttps://www.engineersgarage.com/electronic-components/16×2-lcd-module-datasheethttps://www.openhacks.com/uploadsproductos/eone-1602a1.pdfhttps://www.electronics-notes.com/articles/electronic_components/resistors/light-dependent-resistor-ldr.phphttps://cdn-shop.adafruit.com/datasheets/PCA9685.pdfhttps://www.raspberrypi.org/documentation/hardware/camera/README.mdwww.electronicoscaldas.com/datasheet/MG996R_Tower-Pro.pdfhttps://www.adafruit.comhttps://www.arduino.cchttps://www.raspberrypi.org
