Energy Informatics

Table 2 Evaluated hyperparameter for the different model architectures

From: Estimating time-delayed variables using transformer-based soft sensors

	PV-Data	Inv-Data
Multilayer Perceptron
Batch size	16, 32, 64, 128, 256	16, 32, 64, 128, 256, 512
Number of layers	1, 2, 3, 4	1, 2, 3, 4
Layer size	128, 256, 512, 1024	128, 256, 512, 1024
Dropout	0.1, 0.2	0.1, 0.2
Long Shot-term Memory
Batch size	16, 32, 64, 128, 256	16, 32, 64, 128, 256
Number of layers	2, 3, 4	2, 3, 4
Layer size	64, 128, 256, 512	32, 64, 128, 256, 512
Dropout	0.1, 0.2	0.1, 0.2
Time Series Transformer
Batch size	64, 128, 256, 512	64, 128, 256, 512
Model dimension	32, 64, 128	32, 64, 128
Number of attention heads	8, 16, 32, 64	8, 16, 32, 64
Number of encoder blocks	2, 4, 6, 8	2, 4, 6, 8
Feedforward layer size	32, 64, 128, 256	32, 64, 128, 256

The best hyperparameters are bold

Back to article page