This article will be a quick overview of autoencoders.
Autoencoder is a type of neural network, having the same input and output. It works based on unsupervised learning without the need for a target label. The entire network is sub-divided into two major components (1) Encoder and (2) Decoder.
Encoder: The prime job of the encoder unit is to extract the critical features from the input vector(images, text etc), thereby reducing the original feature space.
Decoder: The decoder portion of the network tries to reconstruct the original input from the extracted features.
Learning Process: The entire learning happens by comparing the input and output using squared errors |acutal_input — predicted_input|² and then minimising the cost. In this way, the network learns the critical parts of the input by excluding the noise present.
Autoencoder for Dimensionality Reduction: Earlier, we’ve seen PCA for dimensionality reduction. But the limitation of PCA is its inability to handle non-linear relationships between the input features. As we know, unstructured data such as images and texts are highly non-linear and complex functions, we need another approach. Autoencoder became the go-to choice for reducing the feature space while handling unstructured data. Since they are neural networks that are capable of depicting complex functionalities, during training the complete architecture is used(encoder + decoder).
Once the learning process is completed with minimal loss, the decoder unit is separated out leaving the encoder intact. When a similar input pattern is fed into the learnt autoencoder network, the details present in the input can be depicted with smaller dimensions.
Autoencoder as Anomaly Detector: Another popular application of autoencoder is in the area of outlier detection. For instance, the goal is to identify unusual bank transactions. Below are the steps for the entire anomaly detection process.
Step1: Gather all possible normal bank transaction details for training.
Step2: Train the autoencoder with the normal transactions, by reducing the loss.
Step3: Keep the entire autoencoder setup for testing, this includes having both encoder and decoder units.
Step4(testing): Scenario1 — normal bank transaction as input, the output is predicted with the input. The output from the decoder unit is compared with the input, the error is minimum and so it is not an outlier.
Step5(testing): Scenario2 — undesirable bank transaction as input, the output is predicted with the input. The output from the decoder unit is compared with the input, the error is maximum and the input transaction is considered as an anomaly.
Step6(final observation): Since the network is trained on normal transactions, it gave a minimum error for the normal ones and a higher error for unusual transactions.