AI, MM-AQI, a member of the MM-sensing family that can estimate the current air pollution level (i.e., PM2.5) using lifelog images. Lifelog images are a set of images captured periodically for a long time by a personal camera. MM-AQI hypothesizes that lifelog images may contain information that correlates to air pollution. Hence, we can use only lifelog images, plus position and time data, to predict air pollution. e scenario is that a user takes a picture with his/her smartphone, or a personal camera that can connect to the smartphone, the crossmodal AI installed in the smartphone will estimate the current air pollution level. First, we use multimodal datasets collected from air pollution stations, mobile devices, and lifelog cameras over a particular area to analyze the correlation between the surrounding environment (e.g., human activities, weather, natural disasters) and air pollution. is step helps us de-cide which features extracted from images can correlate with the air quality index. In other words, we discover the cross attention between image features and air quality index values. en, we build a crossmodal AI to predict the air quality index using only lifelog images. at meant, we need crossmodal translation to translate implicit human activities, weather, and natural disasters from lifelogging images to air quality index values. Figure 2 illustrates the results of correlation analysis between the surrounding environment and air pollution. We recognize that lifelog images and air pollution captured and measured at the same spatiotemporal dimension can map from one to another and back and represent them in joint representation space. Hence, we modify the MM-sensing general structure to create the MM-AQI crossmo-dal AI to predict air pollution using lifelog images, as depicted in Fig. 3. We keep the data-preprocessing, joint representation space, and bidirectional mapping compo-nents. In contrast, the multimodal space component is replaced by the prior knowledge created by learning the correlation among images and AQI at the same place over time. Based on the correlation analysis on a historical da-taset of both lifelog images and AQI, we can build the prior knowledge to help us design global and domain-specic features. As shown in Fig. 2, the area features such as the green zone, dirty zone, street zone, and sidewalk zone, and object features such as vehicles, trees, pedestrians, and haze have a tight correlation with the uctuation of AQI value. Hence, we take these features as domain-spe-cic while using deep learning embedding vectors as ig. F2 The correlation between the surrounding environment (e.g., human activities, natural disasters) and air pollution [13]ig. F3 MM-AQI: The application-wise design162 情報通信研究機構研究報告 Vol.68 No.2 （2022）4 スマートデータ利活用基盤技術
元のページ ../index.html#168