Computer vision systems in unmanned aerial vehicle: a review

Abstract

Inspections in areas of difficult access or hostile to the human, pattern recognition, surveillance and monitoring, are some of the many applications in with Unmanned Aerial Vehicles (UAV), can be a solution, opening up new perspectives for the use of this technology. The navigation and the position of the UAVs can be made by autonomous method through the computational vision, which is a technology of construction of artificial systems capable of read information from images or any multidimensional data and making decisions. This work presents a review of the use of computer vision systems by UAVs, with a focus on its many applications. The main objective is to analyze the latest technologies used for the development of computer vision in UAVs, through the tools of data search, information storage and, mainly, processing and analysis of data. The researches encompasses a publication of recent works, 2011 onwards, from the Science Direct portal. For each work were analyzed the objectives, methodology and results. Based in this analysis, was made a comparison between the techniques and their challenges. From this, future outlook scenarios of UAVs using computational vision are mentioned.

References

ABDESSAMEUD, A.; JANABI-SHARIFI, F.Image-based tracking control of vtol unmanned erialvehicles.Automatica, Elsevier, v. 53, p. 111–119,2015.

ADAMS, S. M.; FRIEDLAND, C. J. A survey of unmanned aerial vehicle (UAV) usage for imagery collection in disaster research and management. In:9th International Workshop on Remote Sensing for disaster Response. [S.l.: s.n.], 2011. v. 8.

AL-KAFF, A.; MORENO, F. M.; JOSÉ, L. J. S.;GARCÍA, F.; MARTÍN, D.; ESCALERA, A. de la;NIEVA, A.; GARCÉA, J. L. M. Vbii-uav: vision-basedinfrastructure inspection-uav. In: SPRINGER.World Conference on Information Systems and Technologies. [S.l.], 2017. p. 221–231.

ALEXANDRIA, A. R. de; CORTEZ, P. C.; FELIX,J. H. da S.; OLIVEIRA, T. M. de; GIRAO, A. M.;FROTA, J. B. B.; ALMEIDA, J. An ocr system fornumerals applied to energy meters.IEEE LatinAmerica Transactions, IEEE, v. 12, n. 6, p. 957–964,2014.

ALVES, T. de S.; OLIVEIRA, C. S. de; SANIN, C.;SZCZERBICKI, E. From knowledge-based vision systems to cognitive vision systems: A review.Procedia Computer Science, Elsevier, v. 126, p.1855–1864, 2018.

ANDREOPOULOS, A.; TSOTSOS, J. K. 50 years of object recognition: Directions forward.Computervision and image understanding, Elsevier, v. 117,n. 8, p. 827–891, 2013.

ASL, H. J.; YOON, J. Adaptive vision-based control of an unmanned aerial vehicle without linear velocity measurements. ISA transactions, Elsevier, v. 65, p.296–306, 2016.

AYDIN, M.; KU ̆GU, E. Safe landing site detection using srtm data for the unmanned aerial vehicles.In: IEEE.2016 24th Signal Processing and Communication Application Conference (SIU).[S.l.], 2016. p. 2125–2128.

BOUGHRARA, H.; CHTOUROU, M.; AMAR,C. B.; CHEN, L. Facial expression recognition based on an mlp neural network using constructive training algorithm.Multimedia Tools and Applications, Springer, v. 75, n. 2, p. 709–731, 2016.

CARVALHO, N. C. R. L.Referenciamento deimagens aéreas utilizando dados de navegação paraconstrução automática de mosaico de imagens. Dissertação (Master in Engineering) — Instituto Tecnológico de Aeronáutica - ITA, São Paulo, Brazil,2014.

CHAO, H.; GU, Y.; NAPOLITANO, M. A survey of optical flow techniques for robotics navigation applications.Journal of Intelligent & RoboticSystems, Springer, v. 73, n. 1-4, p. 361–372, 2014.

CHOWDHARY, G.; JOHNSON, E. N.; MAGREE,D.; WU, A.; SHEIN, A. Gps-denied indoor and outdoor monocular vision aided navigation and control of unmanned aircraft. Journal of Field Robotics,v. 30, n. 3, p. 415–438, 2013. Disponível em: .

CLOTHIER, R. A.; WILLIAMS, B. P.; FULTON,N. L. Structuring the safety case for unmanned aircraft system operations in non-segregated airspace. Safety Science, Elsevier, v. 79, p. 213–228, 2015.

COUTARD, L.; CHAUMETTE, F. Visual detection and 3d model-based tracking for landing on an aircraft carrier. In: IEEE.2011 IEEE InternationalConference on Robotics and Automation. [S.l.],2011. p. 1746–1751.

COUTARD, L.; CHAUMETTE, F.; PFLIMLIN,J.-M. Automatic landing on aircraft carrier by visual serving. In: IEEE.2011 IEEE/RSJ InternationalConference on Intelligent Robots and Systems.[S.l.], 2011. p. 2843–2848.

EL-BAZ, A.; ELNAKIB, A.; EL-GHAR, M. A.;GIMEL’FARB, G.; FALK, R.; FARAG, A. Automatic detection of 2d and 3d lung nodules in chest spiral CT scans.International journal of biomedical imaging, Hindawi, v. 2013, 2013.

ERESEN, A.; ̇IMAMO ̆GLU, N.; EFE, M. Ö.Autonomous quadrotor flight with vision-based obstacle avoidance in virtual environment.ExpertSystems with Applications, Elsevier, v. 39, n. 1, p.894–905, 2012.

FAN, F.; MA, Q.; GE, J.; PENG, Q.; RILEY, W. W.;TANG, S. Prediction of texture characteristics from extrusion food surface images using a computer vision system and artificial neural networks.Journal of food engineering, Elsevier, v. 118, n. 4, p. 426–433, 2013.

FLORES, G.; ZHOU, S.; LOZANO, R.; CASTILLO,P. A vision and GPS-based real-time trajectory planning for a mav in unknown and low-sunlight environments. Journal of Intelligent & Robotic Systems, Springer,v. 74, n. 1-2, p. 59–67, 2014.

GARCIA-PULIDO, J.; PAJARES, G.; DORMIDO,S.; CRUZ, J. M. de la. Recognition of a landing platform for unmanned aerial vehicles by using computer vision-based techniques.Expert Systems with Applications, Elsevier, v. 76, p. 152–165, 2017.

GHAMISI, P.; COUCEIRO, M. S.; BENEDIKTSSON,J. A.; FERREIRA, N. M. An efficient method for segmentation of images based on fractional calculus and natural selection. Expert Systems with Applications, Elsevier, v. 39, n. 16, p. 12407–12417,2012.

HE, S.; YANG, Q.; LAU, R. W.; WANG, J.; YANG,M.-H. Visual tracking via locality sensitive histograms.In: Proceedings of the IEEE conference on computer vision and pattern recognition. [S.l.: s.n.],2013. p. 2427–2434.

HINZMANN, T.; STASTNY, T.; CADENA, C.;SIEGWART, R.; GILITSCHENSKI, I. Free lsd:Prior-free visual landing site detection for autonomous planes. IEEE Robotics and Automation Letters, IEEE, v. 3, n. 3, p. 2545–2552, 2018.

HU, C.; ZHANG, Z.; YANG, N.; SHIN, H.-S.;TSOURDOS, A. Fuzzy multiobjective cooperative surveillance of multiple UAVs based on distributed predictive control for unknown ground moving target in urban environment. Aerospace Science and technology, Elsevier, v. 84, p. 329–338, 2019.

HUANG, C.-L.; CHEN, J.-J.; CHEN, C.-J.; WU,Y.-G. Geological segmentation on UAV aerial image using shape-based lsm with dominant color. In: IEEE 2016 30th International Conference on AdvancedInformation Networking and ApplicationsWorkshops (WAINA). [S.l.], 2016. p. 928–933.

HUANG, L.; SONG, J.; ZHANG, C.; CAI, G. Design and performance analysis of landmark-based ins/vision navigation system for UAV. Optik, Elsevier, v. 172, p.484–493, 2018.

IQBAL, M.; ZHANG, M.; XUE, B. Improvingclassification on images by extracting and transferring knowledge in genetic programming. In: IEEE.2016IEEE Congress on Evolutionary Computation(CEC). [S.l.], 2016. p. 3582–3589.

JI, Q. Combining knowledge with data for efficient and generalizable visual learning. Pattern recognition letters, Elsevier, 2017.

KAI, W.; CHUNZHEN, S.; YI, J. Research on adaptive guidance technology of UAV ship landing system based on net recovery.Procedia Engineering, Elsevier,v. 99, p. 1027–1034, 2015.

KALJAHI, M. A.; SHIVAKUMARA, P.; IDRIS,M. Y. I.; ANISI, M. H.; LU, T.; BLUMENSTEIN, M.; NOOR, N. M. An automatic zone detection system for safe landing of UAVs.Expert Systems with Applications, Elsevier, v. 122, p. 319–333, 2019.

KIM, H.; KIM, K.; KIM, H. Vision-based object-centric safety assessment using fuzzy inference: Monitoring struck-by accidents with moving objects.Journal of Computing in Civil Engineering, American Society of Civil Engineers, v. 30, n. 4, p.04015075, 2015.

KRAJNÍK, T.; NITSCHE, M.; PEDRE, S.; PˇREUˇCIL,L.; MEJAIL, M. E. A simple visual navigation systemfor an uav. In: IEEE.International Multi-Conferenceon Systems, Sygnals & Devices. [S.l.], 2012. p. 1–6.

LARSSON, F.; FELSBERG, M. Using Fourier descriptors and spatial models for traffic sign recognition. In: SPRINGER. Scandinavian conference on image analysis. [S.l.], 2011. p.238–249.

LI, X. A software scheme for UAV's safe landing area discovery.AASRI Procedia, Elsevier, v. 4, p.230–235, 2013.

MARIANANDAM, P. A.; GHOSE, D. Vision based alignment to runway during approach for landing of fixed-wing UAVs. IFAC Proceedings Volumes,Elsevier, v. 47, n. 1, p. 470–476, 2014.

MCFADYEN, A.; MEJIAS, L. A survey of autonomous vision-based see and avoid for unmanned aircraft systems.Progress in Aerospace Sciences, Elsevier, v. 80, p. 1–17, 2016.

MEIER, L.; TANSKANEN, P.; FRAUNDORFER, F.;POLLEFEYS, M. Pixhawk: A system for autonomous flight using onboard computer vision. In: IEEE.2011 IEEE International Conference on Robotics and Automation. [S.l.], 2011. p. 2992–2997.

MENG, Y.; WANG, W.; HAN, H.; BAN, J. A visual/inertial integrated landing guidance method for UAV landing on the ship.Aerospace Science and technology, Elsevier, v. 85, p. 474–480, 2019.

MOREIRA, A.; FERNANDES, A.; SILVA, J. A.;BATISTA, A.; MIRANDA, P. H. The use of a statistical filter and metaheuristics to model and control the DC motor of the mobile robot used on nxp cup. Journal of Mechatronics Engineering, v. 1, n. 1, p.11–20, 2018. ISSN 2595-3230. Disponível em: .

MOTA, F. A. X.; ROCHA, M. X.; RODRIGUES, J. J.P. C.; ALBUQUERQUE, V. H. C.; ALEXANDRIA, A. R. Localization and navigation for autonomous mobile robots using petri nets in indoor environments. IEEE Access, v. 6, p. 31665–31676, 2018. ISSN2169-3536.

PATEL, V. M.; NGUYEN, H. V.; VIDAL, R. Latentspace sparse subspace clustering. In: Proceedings of the IEEE International Conference on ComputerVision. [S.l.: s.n.], 2013. p. 225–232.

PATTERSON, T.; MCCLEAN, S.; MORROW, P.;PARR, G.; LUO, C. Timely autonomous identification of uav safe landing zones.Image and vision computing, Elsevier, v. 32, n. 9, p. 568–578, 2014.

PHUNG, M. D.; QUACH, C. H.; DINH, T. H.; HA,Q. Enhanced discrete particle swarm optimization path planning for uav vision-based surface inspection.Automation in Construction, Elsevier, v. 81, p.25–33, 2017.

QIAN, P.; ZHAO, K.; JIANG, Y.; SU, K.-H.; DENG,Z.; WANG, S.; JR, R. F. M. Knowledge-leveraged transfer fuzzy c-means for texture image segmentation with self-adaptive cluster prototype matching.Knowledge-based systems, Elsevier, v. 130, p. 33–50,2017.

RAJU, P. D. R.; NEELIMA, G. Image segmentation by using histogram thresholding.International Journal of Computer Science Engineering and Technology, Citeseer, v. 2, n. 1, p. 776–779, 2012.

SA JUNIOR, J. J. d. M.; BACKES, A. R. Elmbased signature for texture classification.pattern recognition, Elsevier, v. 51, p. 395–401, 2016.

SHIRZADEH, M.; ASL, H. J.; AMIRKHANI, A.;JALALI, A. A. Vision-based control of a quadrotor utilizing artificial neural networks for tracking of moving targets.Engineering Applications of Artificial Intelligence, Elsevier, v. 58, p. 34–48, 2017.

VALAVANIS, K. P. Unmanned aircraft systems: the current state-of-the-art. Springer Publishing Company, Incorporated, 2016.

VERNON, D.Cognitive Vision Systems: Sampling the Spectrum of Approaches, chap. The space of cognitive vision. [S.l.]: Springer, Heidelberg, 2006.

WANG, S.-H.; ZHAN, T.-M.; CHEN, Y.; ZHANG,Y.; YANG, M.; LU, H.-M.; WANG, H.-N.; LIU, B.; PHILLIPS, P. Multiple sclerosis detection based on biorthogonal wavelet transform, RBF kernel principal component analysis, and logistic regression.IEEEAccess, IEEE, v. 4, p. 7567–7576, 2016.

WARGO, C. A.; CHURCH, G. C.; GLANEUESKI,J.; STROUT, M. Unmanned aircraft systems (uas) research and future analysis. In: IEEE. 2014 IEEE Aerospace Conference. [S.l.], 2014. p. 1–16.

YANG, X.; SHEN, X.; LONG, J.; CHEN, H. An improved median-based otsu image thresholding algorithm. Aasri Procedia, Elsevier, v. 3, p. 468–473,2012.

YU, X.; ZHANG, Y. Sense and avoid technologies with applications to unmanned aircraft systems: Review andprospects. Progress in Aerospace Sciences, Elsevier,v. 74, p. 152–166, 2015.
Published
2019-07-30
How to Cite
MENDES, Odilon Linhares Carvalho; BORILLE, Giovanna Miceli Ronzani. Computer vision systems in unmanned aerial vehicle: a review. Journal of Mechatronics Engineering, [S.l.], v. 2, n. 2, p. 11 - 22, july 2019. ISSN 2595-3230. Available at: <http://jme.ojs.galoa.net.br/index.php/jme/article/view/26>. Date accessed: 16 oct. 2019. doi: https://doi.org/10.21439/jme.v2i2.26.