Generative Adversarial Networks: a systematic review and applications

. Since their introduction in 2014 Generative Adversarial Networks (GANs) have been employed successfully in many areas such as image processing, computer vision, medical imaging, video as well as other disciplines. A large number of review papers have been published, focusing on certain application areas and proposed methods. In this paper, we collected the most recent review papers, organized the collected information according to the application field and we presented the application areas, the GAN architectures that have been applied in each case and summarized the open issues in each area.


Introduction
ML can be described as asubset of artificial intelligence in which a mathematical model based on "training data" is built to make decisions or predictions without being programmed. Machine learning and Neural Networks have been applied for several scientific purposes, such as education applications [63], sign language learning [60], gaming applications [61] and early and objective autism spectrum disorder (ASD) assessment [62]. A basic GAN comprises of a synthetic data generator, a real data source and a discriminator who is trying to classify an item as real or synthetic. In order to improve the performance of GANs in different applications, several variants and have been proposed. In this paper, we have collected the most recent review papers and organized the collected information according to the application field. We present the applications, the GAN architectures that have been applied in each case and summarize the open issues in each area. The paper is organized as follows: Section 2 presents recent review papers that focus on GAN architectures and variants. The most commonly used GAN variants are listed with a brief discussion of each architecture. Section 3 lists the reviews focusing in the area of Computer vision. The most important conclusions of these papers are presented and open research issues in this area are listed. Section 4 contains reviews on image processing applications subdivided to major subfields such as image synthesis, image to image translation super resolution, image in-painting, text to image synthesis. In section 6 are discussed papers related to medical imaging, in section 7 we present reviews in speech processing and in section 8 video related applications of GANs. Section 9 discusses stenography, section 10 contains reviews related to Cyber Security and section 11 addresses the use of GANs in Deefake. Finally section 11 and 12 presents comprehensive review papers in Finance and Manufacturing. Conclusions are given in Section 13. * Corresponding author: asimopou@icloud.com 2 Review papers focusing on GAN architectures and variants.
For researchers who are interested in well organized reviews of GAN structures, architectures and variants, that have been proposed until now, the areas where each variant has been applied and the advantages and limitations that have been reported, the following recent review papers provide in depth analysis and comparative discussion of GANs.

Computer Vision
A large number of review papers are focused on GAN applications related to Computer Vision such as: Image Synthesis,Video Generation, Feature Extraction, Spacio-temporal Data, Imbalance Problems, Person identification, Data enhancement, High-quality sample generation, Domain transfer, Image restoration, AI security. Some of the most recent reviews are: As is reported in the above references, current state-ofthe-art GANs can produce high quality images and diverse images in the computer vision field. Some areas where limited research has been reported and further advances can be introduced, include video applications as well as time-series generation and natural language processing.

Image Processing
The field of image Processing is very extensive, therefore, our study divided it in several more specialized subfields, that are discussed in the following subsections

Image Synthesis
Typical applications of image Synthesis include Facial Expression Synthesis, Synthetic Image Generation 3D Reconstruction, Image Completion, Texture synthesis, Cartoon Generation, Audio-visual emotion Synthesis. Several reviews have been published that focus on advances of GANs in image synthesis and summarize the architectures, their performance and open issues in this subfield. The most recent are listed in Table 3. As is summarized in the listed references, Open Issues in this area are mainly: • Unstable Training amd difficulty in evaluating performance • Suggested future research directions can be in : • Video generation, Facial animation synthesis, 3D face reconstruction.

Image to Image Translation
In this subsection we concentrate on GANs applied to translation of image style from one domain to another. Typical such applications include: Super-Resolution, Style Transfer, Object Transfiguration which aims to detect an object in the source image and another object in the target image with different background, Medical Imaging aiming to provide enhanced post processed images. Two recent reviews are listed in table 4: Multimodal image-to-image translation has not been studied extensively.

Image in-painting
Image in-painting refers to the task of repairing image defects and restoring damaged originals. More information can be found in the articles listed in table 5.

Super Resolution
Here the goal is the generation of high resolution images from low resolution originals

Text to Image Synthesis
Text to image synthesis refers to the task of creating realistic images from plain sentences. Two recent reviews have been published, listed in table 7, that focus in this area of GAN applications.

Medical Imaging
In the area of Medical Images GANs have been applied in two major directions, namely image synthesis and anomaly detection.
In the direction of image synthesis, research is mainly focused on the use of GANs to high resolution imaging methods such as MRI imaging. Table 8 shows some of the recent review papers in image synthesis. Indicative applications of GANs in medical Image Synthesis are: Reconstruction of high-resolution MRI, Image synthesis with different modalities and contrasts, Image Enhancement with preservation of high-frequency information, Creation of multiple data for testing from limited original data. GAN Architectures adopted in medical Imaging include Conditional GANs and CycleGANs GANa are also used for Detecting anomalies in Medical Images in order to assist in the diagnosis process. Extensive research is active is this area and recent reviews of the related research are listed in table 9. The adoption of GANs in medical imaging is still in experimental usage and there are no GAN-based methods adopted for clinical application.

Speech Processing
GANs have also been adopted in the domain of Speech Proceesing. There are numerous publications reporting application of GAN architectures in: Speech synthesis, Speech enhancement and conversion, Noise removal, Fidelity/microphone variability, Voice conversion, Emotional-voice conversion, Pronunciation correction, Intelligibility. A comprehensive review of the above topics is listed in table 10. Wali et al.

2022
A revview of Generative adversarial networks for speech processing.
In the above paper are included the data sets and evaluation metrics used in speech processingGANs, as well as suggestion for research directions for future work such as longer and more realistic speech generation.

Video
The use of GANs in video processing has been applied to areas such as: Video Generation, Speech to Video Synthesis., Text to Video Synthesis., Semantic Map/Missing Region to Video Synthesis, Image to Video Synthesis. Video to Video Synthesis. Current state of the art in video targeted GANs shows limitations is low number of frames and/or low quality frames. Two recent reviews , were an interested researcher can find a comprehensive overview of GANs applied to video are listed in table 11.

Cyber Security
With the vast expansion of computer networks, the needs for massive classification, detection, prediction and optimization problem solving has also increased. Typical related applications are: Intrusion Detection, Password Cracking, Anomaly Generation etc. In this section we suggest the following recent review papers concentrated in Cyber Security applications of GANs.

DeepFake
Deepfake is the technology of using of artificial intelligence to automatically generate or alter images and videos. GANS can be used to generate realistic fake digital data (image, video, speech etc). Future research efforts should focus on further improvement of fake image identification.

Finance
GANs have applied in the area of Finance. Problems such as Time Series Forecasting, Market Prediction, Fine-Tuning of trading models, Portfolio Management, Porfolio Optimization, Time Series Generation, Finance Data Augmentation, Fraud Detection, Detection of Credit Card Fraud can be formulated as big data problems and optimized variants of GANs can be used to provide automated answers to these problems. Several GANs have been applied and optimized to Finance such as FIN-GAN, Conditional GAN (cGAN) , WGAN-GP, Corr-GAN, QuantGAN, MAS-GAN. A recent review of related publications is given below.

Conclusions
GANs have been applied to diverse fields of image, speech and video processing, medical imaging and other fields, where large amounts of data must be processed. In general, GANs have shown considerable advantages such as: • Ability to model partially labelled data.
• Training beyond the available data.
• Limited human involvement in training.
• Efficient generation of samples.
and have presented limitations that need further investigation such as: • Mode collapse • Challenges in certain training models • Difficulty in modeling discrete data (e.g. text) In this paper, we have presented the most recent GAN review papers, we have identified the major areas of research and have organized the review based on these areas. For each major area we have presented the most common GAN architectures applied and we have summarized open issues and future research. We hope that this review will help anyone interested in this research field to acquire a comprehensive view of on going research efforts.