Panoramic Director

Table of Contents

• Introduction
  • Quick start
    • Import the panoramic media files
    • Create a PanoramicStage asset
    • Create a PanoramicExperience asset
    • Add a PanoramicDirector actor
  • Navigation between stages
    • Navigation with a UMG Widget
    • Navigation with a lookup texture
  • Customizing the user interaction
  • Transition effects
    • Cutoff effect
    • Cross-fade effect

Introduction

Class APanoramicDirector is the high level actor to show and interact with a panoramic experience, complete with user navigation. A panoramic experience is composed of a set of stages. Each stage defines which panoramic media to present and how to navigate to other stages.

The transition between stages is automatically activated when the user stares at an hotspot. There are two types of hotspots: custom UMG widgets and areas defined by custom coloured masks. When the user stares at an hotspot, a viewfinder is shown to inform of the imminent navigation. The viewfinder can be customized choosing any UMG widget, or even disabled. Both the pause before activating the navigation and how long the user should stare at the hotspot can be customized. Transition effects (fading and cutoff) between stages can be customized too.

Read 'How-to' for common how-tos.

Quick start

Integrating a panoramic player into your application is quite simple using the APanoramicDirector actor:

1. Import the panoramic media files you want to display.
2. Create a UPanoramicStage asset for each imported panoramic media (to describe how to display the media and how the user can interact with it);
3. Create a UPanoramicExperience asset to describe the experience;
4. Add an APanoramicDirector actor to your scene (to manage the experience at run-time).

For more advanced usages not covered by APanoramicDirector, you can work directly with Panoramic Sphere.

To begin with the APanoramicDirector actor, open the Unreal Engine 5 Editor and create an empty level if you don't have one already.

Import the panoramic media files

You must import in your Unreal Engine 5 project all the panoramic media files (images and videos) you want to display. Please read the section Media Layout for details.

Create a PanoramicStage asset

A UPanoramicStage is used to describe how to display a panoramic media and how the user can interact with it (including which other panoramic stages are reachable from this one).

To create a new UPanoramicStage asset: in the Content Browser, press the Add New button and then select Miscellaneous > Data Asset. Search for PanoramicStage and then click on Select. Choose a name for this stage and then press Enter to confirm.

Create new DataAsset

Double click on the newly create asset to open its Details panel, where you can customize all stage properties.

PanoramicStage base properties

Here you must setup some important properties to control the media asset to be displayed when this stage will be activated:

• Media Type: if the associated media asset is a panoramic Movie or a static panoramic Image;
• Media Layout: if the associated media asset is stereoscopic or mono (see Media Layout for details);
• if Media Type is Image, then:
  • Panoramic Texture: the panoramic image to display;
• if Media Type is Movie, then:
  • Panoramic Media Source: the UE5 media source to play;
  • Playback Loop: enable it to play the video repeatedly and continuously;
  • On Media End Reached Default Behaviour: default action when the media playback reached the end (available only if Playback Loop is disabled);
• Yaw Rotation: an optional yaw rotation angle (in degrees) to control the default orientation of the displayed media (useful to control what part of the panoramic media the user will see in front of him/her when this stage will activate).

Other properties control the set of interactive hotspots the user can interact with to navigate to other stages:

• Navigation Lookup Texture: an optional texture used to define interactive areas, identified by color (see section Navigation below for further details);
• Navigation: the array of interactive hotspots the user can interact with to automatically navigate to other stages (see section Navigation below for further details). Hotspots can be defined in term of UMG widgets (with coordinates relative to the used media) and/or color codes (relative to the Navigation Lookup Texture, if set).

Create a PanoramicExperience asset

A UPanoramicExperience is used to describe and control the overall user experience.

To create a new UPanoramicExperience asset: in the Content Browser*press the *Add New button and then select Miscellaneous > Data Asset. Search for PanoramicExperience and then click on Select. Choose a name for this experience and then press Enter to confirm.

Create new DataAsset

Double click on the newly created asset to open its Details panel, where you can customize the experience properties.

PanoramicExperience properties

At this moment there's only one property to configure:

• Entry Stage: the UPanoramicStage to show when the experience starts (see section Create a PanoramicStage asset).

Add a PanoramicDirector actor

Search for the class PanoramicDirector in the Modes > Place panel and drag it into the level to create a new instance.

Create new PanoramicDirector

Select the newly placed instance (clicking on it in the editor viewport or through the World Outliner) to open its Details panel and configure its properties.

PanoramicDirector base properties

You must now set some required properties:

• Experience: the panoramic experience to play when the actor activates, you can leave it empty to control it at run-time calling the method APanoramicDirector::Play();
• if your experience contains videos, you must set:
  • Media Player: the UE5 Media Player to use to playback videos;
  • Media Texture: the UE5 Media Texture where Media Player renders the video tracks;

You can now press the PIE PLAY button and see the result in real-time.

Other properties of APanoramicDirector are explained below and allow you to customize how the user interact with the experience, how to navigate between stages and the effects to use for transitions.

For details about the Blend Mode option, please read 'Blend mode'.

Navigation between stages

If your experience has two or more stages, you can configure the navigation link between them and APanoramicDirector will take care of present and manage it for you. You can configure how and when navigation between stages takes place configuring the Customizing the user interaction.

A navigation link is configured in the panoramic stage of origin, specifying the destination and how to navigate to it. An empty destination will cause the end of the experience.

PanoramicStage navigation properties

To add a navigation link:

• open the UPanoramicStage asset of origin;
• add a new entry in the Navigation array;
• set the property Destination Stage to the destination UPanoramicStage asset (this will be the stage shown to the user when the navigation will activate); set it to None to cause the experience to exit when the user will interact with this navigation link;
• optionally enable the Yaw Rotation property and set the yaw rotation angle (in degrees) used to re-orient the destination stage when reaching it from this stage. When enabled, this value overrides the corresponding option in the destination stage. This can be used to create a more natural navigation between stages, properly orienting the user when entering a stage from several origins.
• configure the interactive hostspot as explained below.

The last step to configure navigation between stages is to define the interactive area the user will have to interact with to effectively activate the navigation. This interactive area can be defined in two ways:

• using a standard UE5 UMG Widget;
• specifing a Lookup texture and a Lookup color property as explained below.

Navigation with a UMG Widget

This option allows you to place on screen an ordinary UE5 UMG Widget that the user can interact with to activate the navigation. The interactive area is defined by the size of the UMG Widget box.

The UMG Widget used at run-time

Designing a standard UE5 UMG Widget

To configure a navigation link to use an UMG Widget:

• create a standard UE5 UMG Widget;
• set the Widget Class property to the UMG widget you want to display;
• use the Widget Position property to define where - in the panoramic media of this stage of origin - the widget will be displayed. To be able to place the widget precisely, you can input the coordinates using several unit of measure (pixels, UV, yaw/pitch, etc);
• optionally set a Widget Scale (scaling factor).

You can customize how the user will interact with the widgets following the instructions in the section Customizing the user interaction.

Navigation with a lookup texture

This option allows you to defines arbitrary areas in the panoramic image with which the user can interact with.

A Navigation Lookup Texture overlaid on top of the associated panoramic image

To use this navigation method you must:

1. create a navigation lookup texture using custom color codes;
2. configure the navigation links using the above color codes.

Create the Navigation Lookup Texture

You must create a texture with the following characteristics:

• have an aspect ratio of 2:1, with dimensions being power-of-two (e.g. 512x256, 1024x768, etc):
  • the dimensions of this texture can be smaller than the corresponsing panoramic image (saving space and slightly loosing precision);
  • the texture must use a mono layout also if the corresponding media is stereo;
• don't use any transparency (no alpha channel);
• fill with the same RGB color all pixels that must activate the same navigation link (this will define the interactive areas).

This is the Navigation Lookup Texture corresponding to the above example:

An example of Navigation Lookup Texture - colors define different interaction areas

Import the Navigation Lookup Texture

When you import this texture in Unreal Engine 5, you must:

• set Compression Settings to UserInterface2D (RGBA) (compression changes colors);
• set Mip Gen Settings to NoMipmaps;
• disable the sRGB flag (masks must not be Gamma corrected)
• under section Texture > Advanced enable the Never Stream flag

If the navigation does't work as expected, double check you set the flags as described above.

Options to set on the imported Navigation Lookup Texture

Configure the Panoramic Stage

You have to configure the UPanoramicStage asset to use your Navigation Lookup Texture:

1. Open the UPanoramicStage asset for which you are configuring the navigation links;
2. set the property Navigation Lookup Texture to your imported texture.

Then, for each navigation link you want to configure, you must set the correspoding color:

1. identify the navigation link to configure;
2. set the property Lookup Color to the exact same RGB color you used in the Navigation Lookup Texture.

It's done! When the user will interact with the panoramic image (staring at a point or clicking on it), if the corresponding pixel color in the Navigation Lookup Texture will match one of the configured Lookup Color values, then the navigation will activate.

Customizing the user interaction

You can use the Interaction section of the APanoramicDirector to customize how the user will interact with the panoramic experience (mainly to navigate between configured stages, see section Navigation between stages). It has several orthogonal properties, that allow you to configure how to detect a possible interaction and when to activate it.

PanoramicDirector interaction properties

You can configure how to detect an interaction using the following properties:

• Camera Forward as Target Direction: if enabled, use the camera forward vector to check if it intersects some interactive element on the panoramic sphere;
• Mouse Cursor as Target Direction: if enabled, use the mouse cursor on screen to check if it intersects some interactive element on the panoramic sphere;
• (for advanced users) calling the method UPanoramicInteractionComponent::PushCustomWorldTargetDirection() on the APanoramicDirector::InteractionComponent property.

To configure when an interaction will activate (after having been detected), set the following properties:

• Interaction Auto Mode: if enabled, an interaction will start automatically as soon as one interactive element is detected (e.g. with a mouse hover or staring at an hotspot with the camera);
  • Focus Duration: pause in seconds before changing state (e.g. navigating to the new stage);
  • Viewfinder Class: UMG Widget class to show during the focus pause (e.g. an animated cursor). If the widget implements the IPanoramicWidgetViewfinderInterface interface, then you can customize further its behavior (e.g. to sync its animation duration);
• Interaction Input Action Mode: if enabled, you can set Input Action Name to the name of the Input Action Mapping that must activate the interaction (e.g. a mouse click). Look at the official UE5 documentation for details on how to define a custom action mapping.

Transition effects

The APanoramicDirector actor supports several effects that can be customized:

• a cutoff effect used when entering and exiting the panoramic experience, to blend with any surronding 3D environment;
• a cross-fade effect when transitioning between stages.

By default both the effects are disabled and must be configured and turned on if desired.

Cutoff effect

The cutoff effect can be used when entering and exiting the panoramic experience to animate the transition between the surronding 3D environment and the panoramic sphere.

PanoramicDirector cutoff properties

The cutoff effect renders some texels of the panoramic media as transparent, causing the 3D environment behind it to appear. In addition it can also alpha-blend the whole panoramic sphere over time. A texel is rendered as transparent if the corresponding sample in the Cutoff Texture (a greyscale image) has a value lower than a specified threshold. Animating this threshold value (using a custom UE5 Curve asset), we obtain the desired animated cutoff effect.

The Cutoff Curve defines the threshold value for a given instant:

• on X-axis there is the normalized time in [0, 1];
• on Y-axis the interpolation value in [0, 1].

Let d(t) be the threshold value at time t, the media texel at coordinates (u,v) is rendered as transparent if cutoff_texture(u,v) < d(t).

The Cutoff Duration defines the duration in seconds of the effect. Set it to 0 to disable the effect.

The Cutoff Alpha Curve, if set, controls the alpha channel of the effect (from 0 for transparent to 1 for opaque), allowing you to fade it over time.

Setting the relative flags, you can selectively enable/disable the effect:

• on entering and exiting the panoramic experience;
• when transitioning between two panoramic stages.

Cross-fade effect

The cross-fade effect can be used when navigating between panoramic stages. It first fades the origin panoramic media to the given Background Texture and then fades this texture to the destination panoramic media.

PanoramicDirector fading properties

The Fade Curve defines the interpolation value for a given time:

• on X-axis there is the normalized time [0, 1];
• on Y-axis the interpolation value [0, 1].

Let iv(t) be the interpolation value at time t, the final color is equal to media_color * iv(t) + background_color * (1 - iv(t)).

The Fade Duration defines the duration in seconds of the effect. Set it to 0 to disable the effect.

Setting the relative flags, you can selectively enable/disable the effect:

• on entering and exiting the panoramic experience;
• when transitioning between two panoramic stages.