Olfaction and its combination with visual stimuli in the creation of interactive and immersive environments

【下記の日本語訳をご参照ください】

Lead Researchers and authors: Prof Takamichi Nakamoto, Saya Onai, IIR Laboratory for Future Interdisciplinary Research of Science and Technology, School of Engineering, Tokyo Institute of Technology; Nathan Cohen, Tokyo Institute of Technology WRHI Visiting Professor, Central Saint Martins, University of the Arts London.

ARTICLE’S FULL TITLE: Olfaction and its combination with visual stimuli in the creation of interactive and immersive environments, with the potential to enhance personal engagement and well-being – a transdisciplinary collaborative research project integrating science, technology and art.

Since March 2020 we have been working on an olfactory project investigating how smell, in combination with visual stimuli, influences our impression of an environment and how this could impact Virtual Reality (VR) and Augmented Reality (AR) users when creating immersive, interactive scenes.

Smell is a complex medium to work with, posing challenges for developers, technologists and creatives in its identification, application and handling. Invisible yet, to varying degrees, influential in how we interpret our environment, it has been a source of fascination and inspiration for centuries. Samples of aromatic substances are to be found in the graves of early Egyptians and, in refined forms, it has been a rare and sought after commodity across cultures through to the present day.

Smell, or more particularly the odours which form it, can be distributed artificially in different ways. Perfume is worn directly on the body or clothing. Aromatic oil or alcohol based solutions can be sniffed from a simple container or using smelling sticks that absorb the sample. Moved under the nose the aromatic compounds within the odour samples are released into the air and we absorb them through our nasal passages where smell receptors translate the experience to the brain. This requires a simple haptic approach to encountering the smell sample. Alternatives include burning incense, warming essential oils, and the use of aerosol sprays and gel dispensers to disseminate odours more widely within a space.

A high-speed solenoid valve open/close olfactory display setup in the  Nakamoto Laboratory (Credit: authors).
Diagram illustrating use of the high-speed solenoid valve open/close olfactory display set up in the Nakamoto Laboratory. (Credit: authors).

However, to offer a range of odours in a way that is more intimate and corresponds to other stimuli within a controlled and time based environment requires a different approach. Over recent years Takamichi Nakamoto has developed, with his team in the Nakamoto Laboratory, a technical and computerised approach to delivering odour samples, either linked to a headset or in close proximity, enabling their combination with visual and auditory media in immersive digital environments. Many technical difficulties had to be overcome in this construction including how to enable several odour samples to be delivered through one device without their contaminating each other, and how these odours could be linked to different aspects of the immersive environment in which they can be encountered over varying durations of time. The device currently being used for our experiments in the creation of interactive virtual spacio-olfactory games is an high-speed solenoid valve open/close olfactory display.

Scentscape (2019, Nathan Cohen, Reiko Kubota) an interactive olfactory artwork with digital display (Credit: authors).

One area of investigation we are exploring is how smell can encourage memory and enhance narrative association. An earlier example of this may be seen in the boxed artwork Scentscape (2019, Nathan Cohen, Reiko Kubota) where odour samples were presented in small glass screw capped containers which, when handled, triggered sequences of still images on a video screen corresponding to particular places with which the odours are associated. Other objects relating to memories of these places could also be placed within the box, the intention being that the user could combine their own imagery, odour samples and objects to create their own personalised memory box.

The olfactory displays developed by Takamichi Nakamoto offer a more technical and differently immersive approach. In this current research we will be exploring ways in which the combination of imagery and olfaction can create an enhanced narrative experience for the user through the development of scenes that complement and are complemented by odours. This will also take the form of an animated interactive environment that users engage with.

As Saya Onai states in a co-published paper ‘Significant research has already been done in relation to memory and scene recall that can be induced by scent (odours), but information transmission and scene recall by scent alone has not yet been realised effectively. It has been difficult to achieve a common perception of a particular scene from a certain odour due to the influence of prior experience and differences in individual perceptions.’ * In this research, we are seeking ways to enhance scene recollection by combining olfactory, visual, and possibly auditory, stimuli within an immersive environment.

Pair of images displayed sequentially with a 10 second blank screen and odour sample between the first and second versions of the image (Credit: authors).

To better understand the links between smell and memory recall a supervised experiment was conceived by the group working on this research and conducted in the Nakamoto Laboratory by Saya Onai in December 2020. 18 volunteers were divided into 2 separate groups.† Group 1 were presented with the first of 2 related images on a monitor screen, the screen then went blank and an odour related to the image was dispensed through the linked headset for 7 seconds with an overall pause of 10 seconds. Following this the screen displayed the second related image. There were a sequence of 11 sets of visual image pairs with corresponding odours, referred to as ‘scenes’, that all the volunteers experienced consecutively. After viewing all 11 scenes a multiple choice questionnaire was presented asking which odour is the correct one in relation to each scene. For group 1 the questionnaire presented sets of 4 visual images per odour to select from, inviting the participant to identify which image correctly corresponds to the odour for each scene.

Group 1 multiple choice selection for an test odour 
Group 2 multiple choice selection for an test odour (Credit: authors).

Group 2 separately experienced the same test, only the multiple choice options for selecting the correct odour were text based only and not images.

Initial analysis of the experiment’s results reveal mixed rates of identification, with some odour – image associations stronger than others. We will be exploring this further to identify which odour image combinations are more readily identifiable, together with investigating the temporal nature of odour and odour combination in the creation of animated smell scenes that induce narrative association. We will also be considering the aesthetic aspects of the game design required to engage users and enhance their experience of interaction. This will form the basis for the research during April 2021 – March 2022, with the intention to produce a working prototype for display at Siggraph Asia in December 2021.

*香りによる情景想起の基礎的研究  Fundamental Study of Association of Scenes with Scents Nakamoto, T., Onai, S., Iseki, M., Cohen, N. (2021) IEICE General conference

† This was an initial experiment conducted under Covid-19 pandemic restrictions which meant numbers of participants were restricted.

S T A D H I – Science & Technology + Art & Design Hybrid Innovation

This research is supported by the Tokyo Tech World Research Hub Initiative (WRHI), Program of the Institute of Innovative Research (IIR), Tokyo Institute of Technology.

© 2021 Photographs and intellectual content – all rights reserved by the authors.


インタラクティブで没入型の環境の作成における嗅覚と視覚刺激の組み合わせ

2020年3月以来、私たちは嗅覚プロジェクトに取り組んでおり、視覚刺激と組み合わせて、嗅覚が環境の印象にどのように影響し、没入型のインタラクティブなシーンを作成するときに仮想現実(VR)と拡張現実(AR)のユーザーにどのように影響するかを調査しています。

香りは複雑なメディアであり、その識別、応用、処理において、開発者、技術者、クリエイターに課題をもたらします。香りは、香水、アロマオイル、お香、またはエアゾールスプレーやジェルなど、さまざまな方法で人工的に分散させることができます。ただし、制御された時間・環境内で、より緊密で他の刺激に対応する方法でさまざまな香りを提供するには、別のアプローチが必要です。近年、中本高道教授は中本研究所のチームとともに、ヘッドセットにリンクされた、または近接した香りサンプルを拡散するための技術的かつコンピュータ化されたアプローチを開発し、没入型デジタル環境で視覚および聴覚メディアとの組み合わせを可能にしました。

このインタラクティブ嗅覚ディスプレイは、香りがどのように記憶を促し、物語の関連性を高めることができるかを探求するために使用されました。実験は、2020年12月に中本研究室のメンバーによって18人のボランティアに対して実施されました。初期分析結果はさまざまで、強い関連をしめす香りと画像があることが明らかになっています。2021年4月から2022年3月までのさらなる研究は、2021年12月のSIG-GRAPH Asiaで展示するための実用的なプロトタイプを作成することを目指します。