Abstract:

This systematic review focuses on the process of creating photo-realistic and seamless Interactive Immersive Virtual Environments (IIVEs) for use in Virtual Reality (VR) Head Mounted Displays (HMDs) using a combination of 2-dimensional (2D) and 360-degree (360°) images, Photogrammetry, Artificial Intelligence (AI), Machine Learning (ML), inpainting, and outpainting. The IIVE will enable users to navigate in a 3D scene using VR HMDs and controllers.

Current techniques for creating interactive immersive VR experiences rely on one of the following techniques:
1. Specialised cameras to capture 360° images (or video) of real-world locations or processes, stitching software to stitch the spherical images together, and additional software to add contextual information in the form of “hotspots” or “annotations” to the 360° images.
2. Fully rendered 3-dimensional (3D) scenes, created using 3D modeling tools such as 3DS Max, Blender, etc., built using game engines such as Unity or Unreal Engine.

Both techniques have several drawbacks, including (in order as above):
1. Users are not able to interact with the 360° images (or video) other than selecting the hotspots/annotations for contextual or supplementary information, or additionally in the case of video, stopping or starting the video player. Users can not freely navigate around the scene using 360° images or video, instead, they are always viewing the scene from the location and perspective of the camera's current position.
2. Developing fully interactable 3D scenes requires diverse resources and expertise to create an entire 3D scene that sometimes feels “cartoonish” and lacks realism.

This systematic review aims to evaluate the efficacy of employing a combination of techniques, such as photogrammetry, AI inpainting and outpainting, and machine learning (ML), for the development of IIVEs in VR HMDs in comparison to conventional approaches utilising 360° images and fully rendered 3D scenes.

Keywords:

Interactive, immersive, virtual, environment.