\documentclass[twoside,a4paper]{article}
\usepackage{amsmath,amsthm}
\usepackage{amssymb}
\usepackage{subfig}
\usepackage[T1]{fontenc} % Use 8-bit encoding that has 256 glyphs
\usepackage[hmarginratio=1:1,top=32mm,columnsep=20pt]{geometry} % Document margins
\geometry{left=2.5cm,right=2.0cm,top=2.0cm,bottom=2.0cm}
\usepackage{multicol} % Used for the two-column layout of the document
\usepackage{hyperref} % For hyperlinks in the PDF
%\usepackage[hang, small,labelfont=bf,up,textfont=it,up]{caption} % Custom captions under/above floats in tables or figures
%\usepackage{booktabs} % Horizontal rules in tables
\usepackage{lettrine} % The lettrine is the first enlarged letter at the beginning of the text
\usepackage{paralist} % Used for the compactitem environment which makes bullet points with less space between them

\usepackage{abstract} % Allows abstract customization
\renewcommand{\abstractnamefont}{\normalfont\bfseries} % Set the "Abstract" text to bold
\renewcommand{\abstracttextfont}{\normalfont\small\itshape} % Set the abstract itself to small italic text

\usepackage[pagestyles]{titlesec} % Allows customization of titles
%\renewcommand\thesection{\Roman{section}}
%\titleformat{\section}[block]{\large\scshape\centering}{\thesection.}{1em}{} % Change the look of the section titles
\titleformat{\section}[block]{\large\scshape}{\thesection.}{1em}{} % Change the look of the section titles
\titleformat{\subsection}[block]{\normalsize}{\thesubsection.}{1em}{} % Change the look of the section titles
\usepackage[utf8]{vietnam}

\usepackage[mathscr]{eucal}%\setlength{\parskip}{.15in}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%include path for figures
\usepackage{graphicx}
%\graphicspath{{./images}{../Figure}}
%%%%%%%%%%%%%%
\usepackage{indentfirst}
\usepackage{fancyhdr}

\renewpagestyle{plain}{
%	\widenhead[2.5cm][0cm]{0cm}{1.5cm}
	\sethead{}{}{}%
	\setfoot[\bfseries\thepage][][]{}{}{\bfseries\thepage}
}
\pagestyle{plain}
%\usepackage{fixltx2e}
\usepackage{longtable}
%\usepackage[all]{xy}
\usepackage{wrapfig}
\usepackage{subfig}
\usepackage{float}
\usepackage[linesnumbered,ruled,vlined,boxed]{algorithm2e}
\usepackage[numbers,compress,sort]{natbib}
\citestyle{nature}
\usepackage{array}
\usepackage{xcolor}
\usepackage{blindtext}
\captionsetup[table]{name=Tab.,font=small,textfont=it}
\captionsetup[figure]{name=Fig.,font=small,textfont=it}
\newtheorem{theorem}{Theorem}%[]
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\theoremstyle{definition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{remark}[theorem]{Remark}
\newtheorem{assm}{Assumption}
\renewcommand{\baselinestretch}{1.15}
\newcommand{\diag}{\mathrm{diag}}
\newcommand{\argmin}{\mathrm{argmin}}
\DeclareGraphicsExtensions{*.jpg,*.png,*.eps}
\begin{document}

\begin{center}
\fontsize{15pt}{15pt}{\textbf{ỨNG DỤNG MÔ HÌNH YOLO ĐỂ CHẨN ĐOÁN CÁC LOẠI BỆNH VỀ PHỔI DỰA TRÊN ẢNH X-QUANG LỒNG NGỰC}}\\
\vspace{0.5cm}
\fontsize{11pt}{11pt} {\textbf{Lê Phương Thảo$^{1, *}$, Nguyễn Quốc Dương$^{1}$, Đỗ Văn Tuấn$^{1}$}}\\
\vspace{0.4cm}
\fontsize{11pt}{11pt} \textit{$^1$Học viên cao học, Khoa học dữ liệu ứng dụng, Trường Đại học Quy Nhơn} \\
\vspace{0.4cm}		   
\fontsize{11pt}{11pt} {$^*$\textit{Tác giả liên hệ chính. Email: phthao1275@gmail.com}}
\end{center}
	\vskip 0.7cm
	
	
%----------------------------------------------------------------------------------------
%	ABSTRACT
%----------------------------------------------------------------------------------------

\begin{flushleft}
\fontsize{11pt}{10pt}\textbf{TÓM TẮT} 
\end{flushleft}
%\noindent % Dummy abstract text
\hspace{1.5em}
Chụp X-Ray phổi là xét nghiệm cận lâm sàng quan trọng nhất trong chẩn đoán các bệnh lý về phổi. Thông qua kỹ thuật này, bác sĩ sẽ nhanh chóng phát hiện các bất thường trên lồng ngực và lên kế hoạch điều trị bệnh hiệu quả nhất; đồng thời theo dõi tình trạng hồi phục của người bệnh nếu đang trong thời gian điều trị. Mục đích của nghiên cứu này là sử dụng mô hình YOLO để phát hiện các bất thường trên ảnh X-Ray lồng ngực nhằm hỗ trợ bác sĩ chẩn đoán bệnh. Mô hình YOLOv5 và YOLOv7 đã được thử nghiệm để tìm ra mô hình có hiệu suất cao hơn. Chúng tôi sử dụng bộ dữ liệu ảnh X-Ray lồng ngực được cung cấp bởi Vingroup Big Data Institute. Kết quả cho thấy mô hình YOLOv7 có độ chính xác cao hơn so với YOLOv5 dựa trên các thước đo đánh giá như mAP@.5, mAP@.5:mAP@.95, precision và recall. Đồng thời, chúng tôi xây dựng một API để thử nghiệm trực quan hơn cho bài toán này.
\\
\fontsize{10pt}{10pt}\textbf{Từ khóa:} \emph{chest x-ray, x-ray, yolov7, yolov5, vinbigdata} 

\newpage
%----------------------------------------------------------------------------------------
%	TITLE SECTION
%----------------------------------------------------------------------------------------
\title{\vspace{-1.3cm} \hspace*{0.8cm} \fontsize{15pt}{10pt}\textbf{YOLO MODEL APPLICATION TO DIAGNOSE LUNG DISEASES BASED ON CHEST X-RAY IMAGE}} % Article title
\author{\fontsize{11pt}{10pt}\textbf{Phuong-Thao LE$^{1, *}$, Quoc-Duong NGUYEN$^{1}$, Van-Tuan DO$^{1}$}\\ [3mm]% Your name
\normalsize \textit{$^1$Master's Student, Applied Data Science, Quy Nhon University} \\
\normalsize \href{mailto:~}{$^*$\textit{Corresponding author. Email: phthao1275@gmail.com}} % Your email address
\vspace{5mm}
}
\date{}

%----------------------------------------------------------------------------------------



\maketitle % Insert title

%----------------------------------------------------------------------------------------
%	ABSTRACT
%----------------------------------------------------------------------------------------

\begin{flushleft}
\fontsize{11pt}{10pt}\textbf{ABSTRACT} 
\end{flushleft}
%\noindent % Dummy abstract text
\hspace{1.5em}
Chest X-ray is the most important laboratory test in the diagnosis of lung diseases. Through this technique, the doctor will quickly detect abnormalities on the chest and plan the most effective treatment; At the same time, monitor the patient's recovery status during treatment. The purpose of this study is to use the YOLO model to detect abnormalities on chest X-ray images to assist doctors in the diagnosis. Models YOLOv5 and YOLOv7 were tested to find the model with higher performance. We use the chest X-Ray image dataset provided by Vingroup Big Data Institute. The results show that the YOLOv7 model has higher accuracy than YOLOv5 based on evaluation measures such as mAP@.5, mAP@.5:mAP@.95, precision and recall. At the same time, we built an API for more intuitive testing of this problem.
 \\
\fontsize{10pt}{10pt}\textbf{Key words:} \emph{chest x-ray, x-ray, yolov7, yolov5, vinbigdata} 
%----------------------------------------------------------------------------------------
%	ARTICLE CONTENTS
%----------------------------------------------------------------------------------------
\begin{multicols}{2}
\section{Introduction}\label{sec:1}
\noindent 
Artificial Intelligence (AI) is having a huge impact on almost all fields of science, especially the medical field. Today, AI is being applied for the detection, diagnosis and early intervention of human diseases. Some studies have used AI to diagnose diseases such as diabetes \cite{1}, corneal pellets \cite{2}, diagnose liver cancer \cite{3}, predict cerebral palsy \cite{4}, ... This is a step forward in the detection and treatment of diseases. AI also makes it easier for healthcare systems to shift focus and resources from cure to prevention.

Recently, lung diseases are increasing and alarming. It can be said that chest X-ray is the most commonly used technique in clinical examination. Chest X-ray is a technique to help check, screen and detect abnormalities in different locations of the lungs. Through the results of chest X-ray, along with other necessary tests, will suggest the diagnostic of lung-related diseases, thereby giving the most appropriate treatment regimen for the patient. However, radiologists face many challenges every day, one of which is taking and diagnosing chest radiographs. There are currently no specifications for their placement on images, which sometimes leads to inexplicable results. In practice, a complex work of reasoning that often requires careful observation and a good knowledge of anatomical, physiological, and pathological principles. These factors increase the difficulty in developing a consistent and automated technique for reading chest X-ray images, while considering all common thoracic diseases \cite{2}. Stemming from these difficulties, a number of studies have used AI to assist physicians in diagnosing coordination-related diseases through chest X-ray images \cite{5,6,7,8,9}. This problem is collectively known as object detection and classification. This is a common problem in computer vision. The goal of object detection is to identify and classify objects that exist in the image.

For the work of object detection in general and abnormal classification on chest X-ray images in particular, YOLO is one of the highly appreciated models. YOLO is a CNN network model used for object detection, recognition and classification. YOLO is created from the combination of convolutional layers and connected layers. In which, the convolutional layers will extract the features of the image, and the full-connected layers will predict that probability and the coordinates of the object. One of the advantages that YOLO brings is that it only uses the entire image information once and predicts the entire object box containing the objects, the model is built in an end-to-end fashion should be trained. purely by gradient descent. Anyone who has ever worked in object recognition has heard of YOLO. The first version \cite{Nguyen2022} came out in 2016  and so far there have been many improved versions of YOLO. In improved versions of YOLO, YOLOv5 and YOLOv7 \cite{yolov7} have yielded amazing results and are classed as state-of-the-art models of object detection. From the above reasons, in this paper, we apply the YOLO model to detect abnormalities on chest X-ray images in order to provide doctors with useful information to support the diagnosis. We compare the performance for the YOLOv5 and YOLOv7 models to choose the model with the better results. For the implementation, the dataset  \textquotedblleft VinBigData Chest X-ray Abnormalities Detection\textquotedblright  ~is provided by Vingroup Big Data Institute and is available at kaggle\footnote{ \url{https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection}}.

\section{Related works}\label{sec:2}
Currently, studies on the detection and classification of diseases on chest X-ray images are promoted. In 2017, Pranav Rajpurkar and associates developed an algorithm that can detect pneumonia from chest x-ray films with a level far beyond that of practicing radiologists \cite{12}. Recently, the world has faced the covid-19 pandemic and caused a wide range of respiratory problems, ranging from mild to critical or fatal. Several studies have collected and constructed different datasets on chest X-ray images for the purpose of studying the effects of Covid-19 on the lungs. New Generation Computing magazine has published the XCOVNet \cite{XCOVNet} model to help classify chest X-ray images for early detection of COVID-19. In particular, the Radiological Society of North America organized the RSNA Pneumonia Detection Challenge to build an algorithm to detect signs of pneumonia on chest X-ray images. Therefore, building AI models to assist specialists in making diagnoses is extremely necessary and has practical significance.

Stemming from that practice, we need a good enough dataset to support building machine learning models. There are many published chest radiograph data sets including ChestX-ray8, ChestX-ray14 \cite{13}, Padchest \cite{14}, CheXpert \cite{Irvin2019} and MIMIC-CXR \cite{16}. Among of those, ChestX-ray14 an extended version of ChestX-ray8, was released by the US National Institutes of Health (NIH). Most of these datasets have disease type labels but do not specify their location on the X-ray film. This hinders the application of machine learning algorithms to detect and localize breast abnormalities. For that reason, Vingroup Big Data Institute released 18,000 images that were manually annotated by a total of 17 experienced radiologists with 22 local labels of rectangles surrounding abnormalities and 6 global labels of suspect diseases \cite{Nguyen2022}. The data set consists of a training set of 15,000 and a test set of 3,000. Details of the data collection and labeling procedures are detailed in the study \cite{Nguyen2022}.
\begin{figure}[H]
	\centering
	\includegraphics[width=7.5cm,height=5cm]{images/label.png}
	\caption{The flow of creating VinDr-CXR dataset}
	\label{fig:label}
\end{figure}

Lung diseases often present as an area on a chest X-ray. There are many machine learning models to help detect pneumonia on X-ray images based on many different object detection models such as CNN, Mask RCNN, Faster R-CNN, YOLOv3 and they both gave pretty good results in some studies on several chest radiograph data sets. In particular, the YOLOv5 version is good and superior to all previous versions. Recently, the emergence of YOLOv7 has become prominent when it defeats all object detection models. In this paper, we reported the performance of the YOLOv5 and YOLOv7 models based on the VinBigdata dataset.
\section{Architecture of the YOLO model}
The first YOLO version got a lot of attention for outperforming state-of-the-art object detectors such as DPM, Fast R-CNN and Faster R-CNN. Also the simplicity of YOLO was quite refreshing for using only 24 convolutional layers and 2 fully connected layers only, unlike the sophisticated R-CNN based architectures. This first successful initial release of YOLO was a launch pad for development of subsequent YOLO versions, so that a new class of object detectors was born. This class of architecture contains 3 main components: backbone, neck and head.
The backbone has the role to extract features from the image, while the neck tries to aggregate the features from different stages from the backbone. In the end the head does the prediction. Usually the backbone has a quite simple build like ResNet-101 and Darknet-53. For feature aggregation SPP and FPN are a popular choice.

There are now countless versions of YOLO developed by different research groups. Of these, YOLOv5 by ultralytics because it quickly exhibits good performance, is written in a popular framework like PyTorch, and provides support for logging and debugging tools. Other versions, such as YOLOv4, YOLOX and YOLOR are some interesting candidates, but each has its drawbacks. YOLOv4 has similar performance to YOLOv5, however it has longer training time and very rudimentary logging system (does not support tools like Tensorboard). Furthermore, the YOLOv7 algorithm is making a big splash in the computer vision and machine learning communities. The official YOLOv7 article titled “YOLOv7: Trainable Free Bags Set the Most Advanced Technology for Real-Time Object Detectors” was published in July 2022 by Chien-Yao Wang, Alexey Bochkovskiy and Hong-Yuan Mark Liao. The YOLOv7 research paper became extremely popular in just a few days. The YOLOv7 algorithm surpasses all object detection models and previous YOLO versions in both speed and accuracy. It saves on hardware costs and can be trained much faster on small data sets without any pre-trained weights. We will compare the performance of these two models based on the chest-xray dataset.

\section{Evaluation metrics}
YOLO detectors generally use a distinctive format that consists of 5 mandatory and 1 optional values: \textit{x, y, w, h, class, conf (optional)}. The first 2 values \textit{(x, y)} are the coordinates of the center of the bounding box normalized with respect to the width and height of the image. The next values \textit{w,h} are the width and height of the bounding box, also normalized with respect to the width and height of the image. The next value \textit{class} is the class id. Finally, the value \textit{conf} is used only for detected bounding box to store the confidence of the detection. If \textit{conf} is not given, then it can be assumed that the bounding box is a ground truth and not a detection. Each version might have a slight variation of this notation format, but conceptually they are all the same. 

In this section, we provide some metrics to evaluate the model. The first key concept is the Intersection over Union (IoU), which expresses, how good the overlap between a detection and ground truth bounding box is. The IoU divides the overlap area by the union area, as visualized in Figure \ref{fig:iou}.
\begin{figure}[H]
	\centering
	\includegraphics[scale=0.15]{images/iou.png}
	\caption{IoU}
	\label{fig:iou}
\end{figure}


The second key concept is the classification of detections in true positive, false positive, false negative. This can be done by taking each detection and seeing, if it has a high IoU with a ground truth bounding box. For a high IoU (i.e. $IoU \geq 0.5$) , then the detection is considered a true positive, else it's a false postive. If a ground truth bounding box is not overlapped by any detections, then the respective missing detection is classified as false negative. The metrics of precision and recall can be defined:
$$p(c,i) = \frac{TP}{TP + FP},$$
$$r(c,i) = \frac{TP}{TP + FN},$$
where:
\begin{itemize}
	\item[-]{TP = true positives count},
	\item[-]{FP = false positives count},
	\item[-]{FN = false negatives count},
	\item[-]{c = threshold for the confidence of the detections},
	\item[-]{i = threshold for the IoU values}.
\end{itemize}

The $c$ parameter can be adjusted to improve the recall or precision, which makes sometimes the comparison between two models unclear. Setting the same $conf$ in order to compare two models seems a good approach, but this naive approach might not work, if the models have very different distributions of the confidence values e.g. one model is overconfident and the other one is underconfident. In YOLOv5 and YOLOv7, this problem was solved by calculating for each model the precision and recall based on the confidence that maximizes the F1-score:
$$F1(c,i) = 2 * \frac{ p(c,i) * r(c,i)}{p(c,i) + r(c,i)}.$$

This way a manipulation of the threshold of the confidence values can be avoided and therefor a fairer comparison between models is guaranteed. One caveat of this approach is the assumption that precision and recall have the same importance. Another caveat of this approach is that it reduces the performance of the model to a single threshold value, but the mean average precison ($mAP$) can be used to express a more general performance. \\

As the name suggests, the $mAP$ is based on the average precision ($AP$), which is calculated indivudually for each class. The AP for a class $C$ and a IoU threshold $i$ can be calculated as follows:
$$AP_C(i) = \sum_{c}{(r(c_{n},i) - r({c_{n-1},i}))p(c_n, i)}.$$
where $c_n$ are ordered values of confidence thresholds. Finally the mAP is expressed as:
$$mAP(i) = \sum_{C \in classes}{AP_C(i)}.$$
The IoU threshold can be obviously be lowered to increase the mAP, but usually models are compared for at a threshold of 0.5. A final metric is the average mAP, usually denoted as $mAP@[0.5:0.95]$. This is just the average $mAP$ for $IoU$ thresholds ranged in 0.5 to 0.95 with a 0.05 step size.

\section{Experimental Results}\label{sec3}
\subsection{Data Processing}
The dataset used consisted of 18000 thoracic x-ray image files, divided into 2 sets of training and testing with the corresponding number of images 15000 and 3000. Each image is read by leading radiologists in Vietnam. For training data, each scan was independently labeled by three doctors. Meanwhile, for the evaluation data, each scan was labeled by a panel of 5 doctors. We used a dataset that was converted from DICOM to jpg format and kept the original size of the image\footnote{\url{https://www.kaggle.com/datasets/awsaf49/vinbigdata-original-image-dataset}}. The dataset includes the following labels: 0 - Aortic enlargement; 1 – Atelectasis; 2 – Calcification; 3 – Cardiomegaly; 4 – Consolidation; 5 – ILD; 6 – Infiltration; 7 - Lung Opacity; 8 – Node/Mass; 9 - Other lesion; 10 - Pleural effusion; 11 - Pleural thickening; 12 – Pneumothorax; 13 - Pulmonary fibrosis; 14 - "No finding". Labels correspond to lung diseases, details of which can be found at \cite{Nguyen2022}. Figure \ref{fig:imbalance} shows us that the number of each label in the dataset has a very large imbalance. Figure \ref{fig:sample} illustrates some of its items.
\begin{figure}[H]
	\centering
	\includegraphics[scale=0.4]{images/statis.png}
	\caption{Annotations per class}
	\label{fig:imbalance}
\end{figure}
\begin{figure}[H]
	\centering
	\includegraphics[scale=0.15]{images/__results___42_0.png}
	\caption{Some illustrations of the data}
	\label{fig:sample}
\end{figure}

As we introduced in section \ref{sec:2}, each image is independently labeled by 3 doctors, so there can be duplicate predictions on the same area in the image. This will appear to have two predictions with the same label with 2 overlapping prediction boxes with $IOU\geq5$. Both of these boxes are correct predictions because they were annotated independently by 2 radiologists. The model returns 2 output boxes corresponding to the labels. In this case, we use NMS - Non-maximum Suppression to remove 1 cell in the output even though it is the correct result. When using YOLO, and for train data, merging the boxes removes the doctor's attributes and takes the average of the duplicate boxes. The decision whether to merge the boxes will be based on the IoU - Intersection over Union index with a good threshold i.e. $IOU\geq5$. Figure \ref{overlap} shows the bounding boxes before and after pooling. Then we normalize the box coordinates from 0 to 1.
\begin{figure}[H]
\centering
\includegraphics[scale=0.2]{images/overlap.png} 
\caption{Take the average of the coordinates using IOU}
\label{overlap}
\end{figure}

\subsection{Model Training}
Two models were trained with the same number of epochs of 200 and run separately to monitor the effects of the models under the same conditions: google colab pro with GPU A100-SXM4-40GB. They have been tested on identical test/test datasets. We use the mechanism to stop training early when the last 100 epochs have not improved results. The YOLOv5 model stopped training on its own at epoch 171 after 2,938 hours of training, and the YOLOv7 model was trained with 200 epochs after 4,853 hours. From Figure \ref{results} (left), it seems that the best results observed at epoch 70 for YOLOv5.

Based on the training results of the YOLOv7 model in Figure \ref{results} (right), we see that the loss function of val objectness has not converged well. This may be because the learning rate is still quite large. In the future, we will reduce the learning rate for the YOLOv7 model to better converge the model and avoid overfiting.

Table \ref{table} summarizes the results of the two models. We see that the mAP@.5, mAP@.5:mAP@.95, Precision and Recall measures of the YOLOv7 model are all higher than those of the YOLOv5 model. This may initially indicate that the YOLOv7 model exhibits better performance than the YOLOv5 model. In terms of training interval for each epoch, the YOLOv5 model gives faster training results than YOLOv7.
\end{multicols}
\begin{table}[!h]
	\centering
	\caption{The result of the experiments}
	\label{table}
	\begin{tabular}{|c|c|c|c|c|c|}
		\hline
		Result & mAP@.5 & mAP@.5:mAP@.95 & Precision & Recall & Training time per epoch (mins) \\
		\hline
		YOLOv5	& 0.377 & 0.182 & 0.443 & 0.397 & 0.9\\
		\hline
		YOLOv7 & 0.421 & 0.208 & 0.466 & 0.461 & 1.24\\
		\hline
	\end{tabular}
\end{table}

\begin{figure}[ht]
	\begin{minipage}[b]{0.5\linewidth}
			\centering
		\includegraphics[width=7cm,height=5cm]{images/results_yolov5.png} 
	\end{minipage} 
	\begin{minipage}[b]{0.5\linewidth}
			\centering
		\includegraphics[width=7cm,height=5cm]{images/results_yolov7.png} 
	\end{minipage} 
\caption{Training results of YOLOv5 and YOLOv7 models}
\label{results} 
\end{figure}

\begin{multicols}{2}
Based on results of training, we choose the YOLOv7 model and build an API interface written in flask library to demo the results of predicting a new image. Our entire implementation is shown in Figure \ref{fig:workflow}. The API will take as input a chest X-ray image and return abnormal detection results including disease label, bounding box position and diagnostic confidence.

The code captures the entire process of data analysis, data processing, model training, and API implementation from the results of the best model found at \url{https://github.com/nguyenquocduongqnu/Chest-X-Ray-Diagnosis}.
\begin{figure}[H]
\centering
\includegraphics[scale=0.1]{images/workflow.png}
\caption{Workflow}
\label{fig:workflow}
\end{figure}
\section{Conclusion}\label{sec5}
In this study, the YOLOv7 model gave better results than YOLOv5 based on performance evaluation measures. We have implemented an API application to test the results of the problem. However, the mAP@.5 on both models is still not high. In the future, we need to adjust the learning rate for the YOLOv7 model lower so that the model can learn better, consider and handle the data imbalance for better diagnostic results of the model.
\section*{Acknowledgment}
Phuong-Thao LE was funded by Vingroup JSC and supported by the Master, PhD Scholarship Programme of Vingroup Innovation Foundation (VINIF), Institute of Big Data, code \emph{VINIF.2021.ThS.QN.05}. The authors thank Dr. Van-Vu NGUYEN for his helpful remark as well as many valuable comments related to our results.
%\section*{}
\renewcommand{\refname}{References}
\bibliographystyle{mybibstyle}
\bibliography{refs}
%\nocite{*}
\hspace{0.3cm}
%----------------------------------------------------------------------------------------
\end{multicols}

\end{document}
