Journal of Intelligent Computing Jaka Demšar, Zoran Bosni, Igor Kononenko 10 4 2019 https://doi.org/10.6025/jic/2019/10/4/121-127 http://www.dline.info/jic/fulltext/v10n4/jicv10n4_1.pdf The temporal dimension that is ever more prevalent in data makes the data stream mining (incremental learning) an important field of machine learning. In addition to accurate predictions, explanations of models and examples are a crucial component as they provide insight into model’s decision and lessen its black box nature, thus increasing the user’s trust. Proper visual representation of data is also very relevant to user’s understanding — visualization is often utilised in machine learning since it shifts the balance between perception and cognition to take fuller advantage of the brain’s abilities. In this paper we review visualisation in incremental setting and devise an improved version of an existing visualisation of explanations of incremental models. We discuss the detection of concept drift in data streams and experiment with a novel detection method that uses the stream of model’s explanations to determine the places of change in the data domain.