Measuring the dynamics of camouflage in natural scenes using convolutional neural networks

Natural scenes are typically highly heterogeneous, making it difficult to assess camouflage effectiveness for moving objects since their local contrast varies with their momentary position. Camouflage performance is usually assessed through visual search and detection experiments involving human observers. However, such studies are time-consuming and expensive since they involve many observers and repetitions. Here, we show that a (state-of-the-art) convolutional neural network (YOLO) can be applied to measure the camouflage effectiveness of stationary and moving persons in a natural scene. The network is trained on human observer data. For each detection, it also provides the probability that the detected object is correctly classified as a person, which is directly related to camouflage effectiveness: more effective camouflage yields lower classification probabilities. By plotting the classification probability as a function of a person’s position in the scene, a ‘camouflage efficiency heatmap’ is obtained, that reflects the variation of camouflage effectiveness over the scene. Such a heatmap can for instance be used to determine locations in a scene where the person is most effectively camouflaged. Also, YOLO can be applied dynamically during a scene traversal, providing real-time feedback on a person’s detectability. We compared the YOLO-predicted classification probability for a soldier in camouflage clothing moving through a rural scene to human performance. Camouflage effectiveness predicted by YOLO agrees closely with human observer assessment. Thus, YOLO appears an efficient tool for the assessment of camouflage of static as well as dynamic objects.