Applications of Mathematics, Vol. 66, No. 2, pp. 287-317, 2021
Variational Gaussian process for optimal sensor placement
Gabor Tajnafoi, Rossella Arcucci, Laetitia Mottet, Carolanne Vouriot, Miguel Molina-Solana, Christopher Pain, Yi-Ke Guo
Received November 16, 2019. Published online February 12, 2021.
Abstract: Sensor placement is an optimisation problem that has recently gained great relevance. In order to achieve accurate online updates of a predictive model, sensors are used to provide observations. When sensor location is optimally selected, the predictive model can greatly reduce its internal errors. A greedy-selection algorithm is used for locating these optimal spatial locations from a numerical embedded space. A novel architecture for solving this big data problem is proposed, relying on a variational Gaussian process. The generalisation of the model is further improved via the preconditioning of its inputs: Masked Autoregressive Flows are implemented to learn nonlinear, invertible transformations of the conditionally modelled spatial features. Finally, a global optimisation strategy extending the Mutual Information-based optimisation and fine-tuning of the selected optimal location is proposed. The methodology is parallelised to speed up the computational time, making these tools very fast despite the high complexity associated with both spatial modelling and placement tasks. The model is applied to a real three-dimensional test case considering a room within the Clarence Centre building located in Elephant and Castle, London, UK.
Keywords: sensor placement; variational Gaussian process; mutual information
References: [1] K. Abhishek, M. P. Singh, S. Ghosh, A. Anand: Weather forecasting model using artificial neural network. Procedia Technology 4 (2012), 311-318. DOI 10.1016/j.protcy.2012.05.047
[2] Applied Modelling, Computation Group: Fluidity manual (Version 4.1). Available at https://figshare.com/articles/Fluidity_Manual/1387713 (2015), 329 pages.
[3] R. Arcucci, L. D'Amore, J. Pistoia, R. Toumi, A. Murli: On the variational data assimilation problem solving and sensitivity analysis. J. Comput. Phys. 335 (2017), 311-326. DOI 10.1016/j.jcp.2017.01.034 | MR 3612500 | Zbl 1375.49036
[4] R. Arcucci, D. McIlwraith, Y.-K. Guo: Scalable weak constraint Gaussian processes. Computational Science - ICCS 2019. Lecture Notes in Computer Science 11539. Springer, Cham (2019), 111-125. DOI 10.1007/978-3-030-22747-0_9 | MR 3976280
[5] R. Arcucci, L. Mottet, C. Pain, Y.-K. Guo: Optimal reduced space for variational data assimilation. J. Comput. Phys. 379 (2019), 51-69. DOI 10.1016/j.jcp.2018.10.042 | MR 3881150
[6] E. Aristodemou, R. Arcucci, L. Mottet, A. Robins, C. Pain, Y.-K. Guo: Enhancing CFD-LES air pollution prediction accuracy using data assimilation. Building and Environment 165 (2019), Article ID 106383, 15 pages. DOI 10.1016/j.buildenv.2019.106383
[7] M. J. Beal: Variational Algorithms for Approximate Bayesian Inference: A Thesis Submitted for the Degree of Doctor of Philosophy of the University of London. University of London, London (2003).
[8] J. H. T. Bentham: Microscale Modelling of Air Flow and Pollutant Dispersion in the Urban Environment: Doctoral Thesis. University of London, London (2004).
[9] D. M. Blei, A. Kucukelbir, J. D. McAuliffe: Variational inference: A review for statisticians. J. Am. Stat. Assoc. 112 (2017), 859-877. DOI 10.1080/01621459.2017.1285773 | MR 3671776
[10] B. Bócsi, P. Hennig, L. Csató, J. Peters: Learning tracking control with forward models. IEEE International Conference on Robotics and Automation (ICRA). IEEE, New York (2012), 259-264. DOI 10.1109/ICRA.2012.6224831
[11] D. Cornford, I. T. Nabney, C. K. I. Williams: Adding constrained discontinuities to Gaussian process models of wind fields. Advances in Neural Information Processing Systems 11 (NIPS 1998). MIT Press, Cambridge (1999), 861-867.
[12] N. Cressie: Statistics for spatial data. Terra Nova 4 (1992), 613-617. DOI 10.1111/j.1365-3121.1992.tb00605.x
[13] L. D'Amore, R. Arcucci, L. Marcellino, A. Murli: A parallel three-dimensional variational data assimilation scheme. Numerical Analysis and Applied Mathematics, ICNAAM 2011. AIP Conference Proceedings 1389. AIP, Melville (2011), 1829-1831. DOI 10.1063/1.3636965 | Zbl 1262.65002
[14] C. Doersch: Tutorial on variational autoencoders. Available at https://arxiv.org/abs/1606.05908 (2016), 23 pages.
[15] T. H. Dur, R. Arcucci, L. Mottet, M. Molina Solana, C. Pain, Y.-K. Guo: Weak constraint Gaussian processes for optimal sensor placement. J. Comput. Sci. 42 (2020), Article ID 101110, 12 pages. DOI 10.1016/j.jocs.2020.101110 | MR 4082342
[16] M. Germain, K. Gregor, I. Murray, H. Larochelle: MADE: Masked Autoencoder for Distribution Estimation. Proc. Mach. Learn. Res. 37 (2015), 881-889.
[17] H. González-Banos: A randomized art-gallery algorithm for sensor placement. SCG'01: Proceedings of the 17th Annual Symposium on Computational Geometry. ACM, New York (2001), 232-240. DOI 10.1145/378583.378674 | Zbl 1375.68139
[18] I. Goodfellow, Y. Bengio, A. Courville: Deep Learning. Adaptive Computation and Machine Learning. MIT Press, Cambridge (2016). MR 3617773 | Zbl 1373.68009
[19] Google Brain Team: TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Available at https://www.tensorflow.org/ (2015). SW 15170
[20] C. Guestrin, A. Krause, A. P. Singh: Near-optimal sensor placements in Gaussian processes. ICML'05: Proceedings of the 22nd International Conference on Machine Learning. ACM, New York (2005), 265-272. DOI 10.1145/1102351.1102385
[21] J. Hagan, A. R. Gillis, J. Chan: Explaining official delinquency: A spatial study of class, conflict and control. Sociological Quarterly 19 (1978), 386-398. DOI 10.1111/j.1533-8525.1978.tb01183.x
[22] J. Hensman, N. Fusi, N. D. Lawrence: Gaussian processes for big data. Available at https://arxiv.org/abs/1309.6835 (2013), 9 pages.
[23] N. Jarrin, S. Benhamadouche, D. Laurence, R. Prosser: A synthetic-eddy-method for generating inflow conditions for large-eddy simulations. Int. J. Heat Fluid Flow 27 (2006), 585-593. DOI 10.1016/j.ijheatfluidflow.2006.02.006
[24] F. J. Kelly, J. C. Fussell: Improving indoor air quality, health and performance within environments where people live, travel, learn and work. Atmospheric Environment 200 (2019), 90-109. DOI 10.1016/j.atmosenv.2018.11.058
[25] D. P. Kingma, M. Welling: Auto-encoding variational Bayes. Available at https://arxiv.org/abs/1312.6114 (2013), 14 pages.
[26] A. Krause, A. Singh, C. Guestrin: Near-optimal sensor placements in Gaussian processes: Theory, efficient algorithms and empirical studies. J. Mach. Learn. Res. 9 (2008), 235-284. Zbl 1225.68192
[27] S. Kullback, R. A. Leibler: On information and sufficiency. Ann. Math. Stat. 22 (1951), 79-86. DOI 10.1214/aoms/1177729694 | MR 0039968 | Zbl 0042.38403
[28] C.-C. Lin, L. L. Wang: Forecasting simulations of indoor environment using data assimilation via an ensemble Kalman filter. Building and Environment 64 (2013), 169-176. DOI 10.1016/j.buildenv.2013.03.008
[29] H. Liu, Y.-S. Ong, X. Shen, J. Cai: When Gaussian process meets big data: A review of scalable GPs. Available at https://arxiv.org/abs/1807.01065 (2018), 20 pages.
[30] D. J. C. MacKay: Introduction to Gaussian processes. Neural Networks and Machine Learning. NATO ASI Series F Computer and Systems Sciences 168. Springer, Berlin (1998), 133-166.
[31] M. I. Mead, O. A. M. Popoola, G. B. Stewart, P. Landshoff, M. Calleja, M. Hayes, J. J. Baldovi, M. W. McLeod, T. F. Hodgson, J. Dicks, A. Lewis, J. Cohen, R. Baron, J. R. Saffell, R. L. Jones: The use of electrochemical sensors for monitoring urban air quality in low-cost, high-density networks. Atmospheric Environment 70 (2013), 186-203. DOI 10.1016/j.atmosenv.2012.11.060
[32] C. C. Pain, A. P. Umpleby, C. R. E. de Oliveira, A. J. H. Goddard: Tetrahedral mesh optimisation and adaptivity for steady-state and transient finite element calculations. Comput. Methods Appl. Mech. Eng. 190 (2001), 3771-3796. DOI 10.1016/S0045-7825(00)00294-2 | Zbl 1008.76041
[33] G. Papamakarios, T. Pavlakou, I. Murray: Masked autoregressive flow for density estimation. Advances in Neural Information Processing Systems 30 (NIPS 2017). MIT Press, Cambridge (2017), 2338-2347.
[34] D. Pavlidis, G. J. Gorman, J. L. M. A. Gomes, C. C. Pain, H. ApSimon: Synthetic-eddy method for urban atmospheric flow modelling. Boundary-Layer Meteorology 136 (2010), 285-299. DOI 10.1007/s10546-010-9508-x
[35] J. Quiñonero-Candela, C. E. Rasmussen: A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6 (2005), 1939-1959. MR 2249877 | Zbl 1222.68282
[36] N. Ramakrishnan, C. Bailey-Kellogg, S. Tadepalliy, V. N. Pandey: Gaussian processes for active data mining of spatial aggregates. Proceedings of the 2005 SIAM International Conference on Data Mining. SIAM, Philadelphia (2005), 427-438. DOI 10.1137/1.9781611972757.38
[37] C. E. Rasmussen: Gaussian processes in machine learning. Advanced Lectures on Machine Learning. Lecture Notes in Computer Science 3176. Springer, Berlin (2003), 63-71. DOI 10.1007/978-3-540-28650-9_4
[38] D. J. Rezende, S. Mohamed: Variational inference with normalizing flows. Available at https://arxiv.org/abs/1505.05770 (2015), 10 pages.
[39] J. Smagorinsky: General circulation experiments with the primitive equations I. The basic experiment. Mon. Wea. Rev. 91 (1963), 99-164. DOI 10.1175/1520-0493(1963)091<0099:GCEWTP>2.3.CO;2
[40] J. Song, S. Fan, W. Lin, L. Mottet, H. Woodward, M. Davies Wykes, R. Arcucci, D. Xiao, J.-E. Debay, H. ApSimon, E. Aristodenou, D. Birch, M. Carpentieri, F. Fang, M. Herzog, G. R. Hunt, R. L. Jones, C. Pain, D. Pavlidis, A. G. Robins, C. A. Short, P. F. Linden: Natural ventilation in cities: The implications of fluid mechanics. Building Research & Information 46 (2018), 809-828. DOI 10.1080/09613218.2018.1468158
[41] M. K. Titsias: Variational learning of inducing variables in sparse Gaussian processes. Proc. Mach. Learn. Res. 5 (2009), 567-574.
[42] M. K. Titsias: Variational Model Selection for Sparse Gaussian Process Regression. Technical report, University of Manchester, Manchester (2009).
[43] V. H. Tran: Copula variational Bayes inference via information geometry. Available at https://arxiv.org/abs/1803.10998 (2018), 23 pages .
[44] D. Tran, R. Ranganath, D. M. Blei: The variational Gaussian process. Available at https://arxiv.org/abs/1511.06499 (2015), 14 pages.
[45] H. Wickham: ggplot2: Elegant Graphics for Data Analysis. Use R! Springer, Cham (2016). DOI 10.1007/978-3-319-24277-4 | Zbl 1397.62006
Affiliations: Gabor Tajnafoi, Rossella Arcucci (corresponding author), Data Science Institute, Department of Computing, Imperial College London, South Kensington Campus, London SW7 2AZ, United Kingdom, e-mail: gabor.tajnafoi18@imperial.ac.uk, gtajnafoi@gmail.com, r.arcucci@imperial.ac.uk; Laetitia Mottet, Applied Modelling and Computation Group, Department of Earth Science & Engineering, Imperial College London, South Kensington Campus, London SW7 2AZ, United Kingdom, e-mail: l.mottet@imperial.ac.uk; Carolanne Vouriot, Department of Civil Engineering, Imperial College London, 58 Princes Gate, Kensington, London SW7 1AL, United Kigdom, e-mail: carolanne.vouriot12@imperial.ac.uk; Miguel Molina-Solana, Department of Computer Science and AI, Universidad de Granada, 18071 Granada, Spain, and Data Science Institute, Department of Computing, Imperial College London, South Kensington Campus, London SW7 2AZ, United Kingdom, e-mail: miguelmolina@ugr.es; Christopher Pain, Department of Earth Science & Engineering, Imperial College London, South Kensington Campus, London SW7 2AZ, United Kingdom, e-mail: c.pain@imperial.ac.uk; Yi-Ke Guo, Data Science Institute, Department of Computing, Imperial College London, South Kensington Campus, London SW7 2AZ, United Kingdom, e-mail: y.guo@imperial.ac.uk
