Manifold Analysis for High-Dimensional Socio-Environmental Surveys

Recent studies on anthropogenic climate change demonstrate a disproportionate effect on agriculture in the Global South and North. Questionnaires have become a common tool to capture the impact of climatic shocks on household agricultural income and consequently on farmers’ adaptation strategies. These questionnaires are high-dimensional and contain data on several aspects of an individual (household) such as spatial and demographic characteristics, socio-economic conditions, farming practices, adaptation choices, and constraints. The extraction of insights from these high-dimensional datasets is far from trivial. Standard tools such as Principal Component Analysis, Factor Analysis, and Regression models are routinely used in such analysis, but they either rely on a pairwise correlation matrix, assume specific (conditional) probability distributions, or assume that the survey data lies in a linear subspace. Recent advances in manifold learning techniques have demonstrated better detection of different behavioural regimes from surveys. This paper uses Bangladesh Climate Change Adaptation Survey data to compare three non-linear manifold techniques: Fisher Information Non-Parametric Embedding (FINE), Diffusion Maps and t-SNE. Using a simulation framework, we show that FINE appears to consistently outperform the other methods except for questionnaires with high multi-partite information. Although not limited by the need to impose a grouping scheme on data, t-SNE and Diffusion Maps require hyperparameter tuning and thus more computational effort, unlike FINE which is non-parametric. Finally, we demonstrate FINE’s ability to detect adaptation regimes and corresponding key drivers from high-dimensional data.