Detecting Android Malware by Mining Enhanced System Call Graphs

Rajif Agung Yunmar*, Sri Suning Kusumawardani, Widyawan Widyawan, Fadi Mohsen

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

22 Downloads (Pure)


The persistent threat of malicious applications targeting Android devices has been growing in numbers and severity. Numerous techniques have been utilized to defend against this thread, including heuristic-based ones, which are able to detect unknown malware. Among the many features that this technique uses are system calls. Researchers have used several representation methods to capture system calls, such as histograms. However, some information may be lost if the system calls as a feature is only represented as a 1-dimensional vector. Graphs can represent the interaction of different system calls in an unusual or suspicious way, which can indicate malicious behavior. This study uses machine learning algorithms to recognize malicious behavior represented in a graph. The system call graph was fed into machine learning algorithms such as AdaBoost, Decision Table, Naïve Bayes, Random Forest, IBk, J48, and Logistic regression. We further employ a series feature selection method to improve detection accuracy and eliminate computational complexity. Our experiment results show that the proposed method has reduced feature dimension to 91.95% and provides 95.32% detection accuracy.

Original languageEnglish
Pages (from-to)28-41
Number of pages14
JournalInternational Journal of Computer Network and Information Security
Issue number2
Publication statusPublished - 1-Apr-2024


  • Android
  • Graph
  • Heuristic-based Detection
  • Machine Learning
  • Malware
  • System Call

Cite this