Autoencoder and Variational Autoencoder

An autoencoder is a neural network that compresses inputs into a low-dimensional representation and reconstructs them, and a variational autoencoder (VAE) learns a probabilistic latent distribution from which new samples can be generated.

오토인코더는 입력을 저차원 표현으로 압축했다가 복원하는 신경망이며, 변분 오토인코더(VAE)는 새로운 표본을 생성할 수 있는 확률적 잠재분포를 학습한다.

How do I learn a compressed representation of my data or detect unusual observations with a neural network?

신경망으로 자료의 압축된 표현을 학습하거나 이상 관측치를 탐지하려면 어떻게 하나요?

An autoencoder learns to reconstruct its input through a bottleneck layer, so the latent codes summarize the data and high reconstruction error flags anomalies. A variational autoencoder adds a probabilistic latent space regularized toward a prior, which fits generating new samples and producing smooth latent representations.

When to use Autoencoder and Variational Autoencoder

• Nonlinear dimensionality reduction of many numeric features
• Anomaly detection using reconstruction error
• Generating synthetic samples or exploring a smooth latent space with a VAE

When to use something else

• Linear structure is sufficient and interpretability matters: use principal component analysis
• The sample is small: use PCA or factor analysis instead of a neural network

Assumptions and requirements

• Large enough sample to train a neural network
• Scaled or normalized inputs
• Choice of latent dimension and architecture validated on held-out data
• Training data that represent normal cases when used for anomaly detection

What the analysis reports

• Training and validation loss curves
• Reconstruction error per observation
• Latent representations (embeddings)
• KL divergence term for a VAE
• Latent space visualization

How to report it

The autoencoder with a [k]-dimensional latent layer reached a validation reconstruction error of [value], and observations above the [value] percentile were flagged as anomalies.

Related methods

• Principal Component Analysis (PCA)
• Isolation Forest
• Generative Adversarial Network

Run Autoencoder and Variational Autoencoder on Quorant