The rapid development of Internet of Things (IoT) devices and their increasing numbers have caused a tremendous increase in network traffic and a wider range of cyber-attacks. This growing trend has complicated the detection process for traditional intrusion detection systems and heightened the challenges faced by these devices, such as imbalanced and large training data. This study presents a cohesive methodology of a series of intelligent techniques to prepare clean and balanced data for training the first (core) layer of a robust hierarchical intrusion detection system. The methodology was built by cleaning and compressing the data using an Autoencoder and preparing a strong latent space for balancing using a hybrid method that combines Grey Wolf Optimization (GWO) with Borderline-SMOTE. Particle Swarm Optimization (PSO) was used to select the most important features that provide the greatest amount of information for training the first layer, which was built using deep learning techniques, and linking them in a hybrid manner that combines a Convolutional Neural Network–Long Short-Term Memory (CNN–LSTM) and the Attention mechanism. The proposed model was evaluated using two different types of datasets: the CICIOT2023 dataset, which is characterized by its large size and significant variation in the number of attacks, and the UNSW-NB15 dataset, which is characterized by its simplicity and less imbalance compared to the first dataset, to prepare and generalize the system across multiple environments. The proposed class showed binary classification results with an accuracy of 0.94, an Area Under the Curve (AUC) of 0.93, an optimized F1-score of 0.338, and a Matthews Correlation Coefficient (MCC) of 0.324 at the best threshold on the CICIoT2023 dataset. It also achieved an accuracy of approximately 0.96, an AUC of 0.985, and an MCC of over 0.82 on the UNSW-NB15 dataset. These results confirmed the construction of a strong and resilient layer, preparing the foundation for a robust hierarchical offside detection system.
