An improved feature extraction algorithm for insider threat using hidden Markov model on user behavior detection

Abstract

Purpose

By using a new feature extraction method on the Cert data set and using a hidden Markov model (HMM) to model and analyze the behavior of users to distinguish whether the behavior is normal within a continuous period.

Design/methodology/approach

Feature extraction of five parts of the time series by rules and sorting in chronological order. Use the obtained features to calculate the probability parameters required by the HMM model and establish a behavior model for each user. When the user has abnormal behavior, the model will return a very low probability value to distinguish between normal and abnormal information.

Findings

Generally, HMM parameters are obtained by supervised learning and unsupervised learning, but the hidden state cannot be clearly defined. When the hidden state is determined according to the data set, the accuracy of the model will be improved.

Originality/value

This paper proposes a new feature extraction method and analysis mode, which determines the shape of the hidden state according to the situation of the data set, making subsequent HMM modeling simple and efficient and in turn improving the accuracy of user behavior detection.

Citation

Ye, X. and Han, M.-M. (2022), "An improved feature extraction algorithm for insider threat using hidden Markov model on user behavior detection", Information and Computer Security, Vol. 30 No. 1, pp. 19-36. https://doi.org/10.1108/ICS-12-2019-0142