4D Ray-Space and Ultra-Wide Area FTV

–Ultra-wide area FTV is an FTV with very wide viewing zone where motion parallax is realized. 4D orthogonal ray-space is analyzed and applied to ultra-wide area FTV. Ray-space of “a group of rays through one point” is derived in 4D orthogonal ray-space. It is extended to obtain ray-space captured by linear arrangement cameras. View generation of ultra-wide area FTV needs rays that are not captured by real cameras. These rays are synthesized by interpolating the captured ray-space so that the intersections of the captured ray-space and the ray-space of rays emitted from a light source have the same color.


INTRODUCTION
FTV (Free-viewpoint Television) [1]- [8] is visual media that transmits all ray information of a 3D space and enables immersive 3D viewing as if we were actually there. Omnidirectional FTV is an FTV with very wide field of view (FOV), that is 360-degree FOV. Ultra-wide area FTV is an FTV with very wide viewing zone that gives motion parallax. FTV was developed based on ray-space representation [9]- [12]. 4D ray-space is needed for immersive 3D viewing with full parallax.
We proposed two types of ray-space, orthogonal ray-space and spherical ray-space. Orthogonal ray-space is used for linear and planar camera arrangements and spherical ray-space is used for circular and spherical camera arrangements. 4D spherical ray-space was analyzed and applied to omnidirectional FTV [13]. In this paper, 4D orthogonal ray-space is analyzed and applied to ultra-wide area FTV. Ray capture and view generation in 4D orthogonal ray-space are presented. Fig. 1 shows the definition of orthogonal ray-space. As shown in Fig. 1, one ray is expressed by intersection (x, y) and direction (θ, φ) on a reference plane. Therefore, one ray is expressed by one point in 4D parameter space (x, y, θ, φ). This parameter space is 4D orthogonal ray-space. Each point in the 4D ray-space has an intensity f of the ray. Therefore, f is expressed as f (x, y, θ, φ).

4D Orthogonal Ray-Space of Rays through One Point
Here, "a group of rays through one point" is considered as shown in Fig. 2. This concept is used for ray capture, view generation and ray synthesis as shown in Fig. 3. Fig. 3(a) shows ray capture by a real camera, and view generation by a virtual camera collecting rays through one point. Fig. 3(b) shows ray synthesis. If the surface of an object is a diffusion surface, one point on the surface emits rays with equal magnitude. Therefore, if one ray is known, other rays can be synthesized.   x and y of a ray with direction (θ, φ) through a point P(X, Y, Z) are given by (1) and (2) give the 4D orthogonal ray-space (x, y, θ, φ) of a group of rays through one point. By changing θ and φ, this group of rays forms a plane in (x, y, θ, φ) space. To visualize it, (x, y, θ) space with fixed φ and (x, y, φ) space with fixed θ are considered in the following.  (1) and (2), the following equations are derived.
For fixed φ, (1), (3) and (4) give the top view, front view and side view of (x, y, θ) space, respectively. The top view is a straight line with slope -Z and both the front and side views are hyperbolic curves in (x, y, tanθ) space. Therefore, a group of rays through one point forms a hyperbolic curve in (x, y, tanθ) space with fixed φ as shown in Fig. 4. Three examples of hyperbolic curves for φ＞0, φ=0 and φ＜0 are shown.

4D Orthogonal Ray-Space Captured by Linear Arrangement Cameras
Let arrange many cameras on a line that is set on the xz-plane (y=0) and parallel to the x-axis. When an omnidirectional camera at P(X c , 0, Z c ) on the line captures a ray with direction (θ, φ) as shown in Fig. 6, x and y are given by y= Zc cosθ tanφ=Z c tanφ 1 + tan 2 θ (6) (5) and (6) give ray-space captured by linear arrangement cameras. By varying X c , (5) and (6) form a hyperbolic cylinder in (x, y, tanθ) space with fixed φ as shown in Fig. 7. They form a plane in (x, y, tanφ) space with fixed θ as shown in Fig. 8. The expressions of the hyperbolic cylinder and the plane are given by y=Z c tanφ 1 + tan 2 θ for any (7) and y= Zc cosθ tanφ for any (8) respectively.

VIEW SYNTHESIS FOR ULTRA-WIDE AREA FTV
4D orthogonal ray-space is applied to ultra-wide area FTV [11]. Ray capture and synthesis for ultra-wide area FTV is shown in Fig. 7. Rays are captured by many cameras on a line (y=0, z=Z c ) and virtual camera views are synthesized from the captured rays. Let one of the rays captured by a virtual camera be (x 0 , y 0 , θ 0 , φ 0 ). This ray is synthesized because it is not captured by real cameras. When z of the light source of this ray is assumed to be Z, the position (X, Y, Z) of the light source is determined. This light source emits rays in various directions. Let ray-space of the emitted rays be (x, y, θ, φ).
Many intersections are obtained by varying θ. Fig. 11 shows the trajectory of this intersection. It is on a horizontal plane of y = ZcY Zc-Z . When Z is correct depth, the intersections have the same color. This color is given to the ray (x 0 , y 0 , θ 0 , φ 0 ). Fig. 10 Captured ray-space, emitted ray-space and their intersection in (x, y, φ) space with fixed θ. Fig. 11 Trajectory of the intersection of captured ray-space and emitted ray-space in (x, y, φ) space by varying θ.
The process of this ray synthesis is summarized in the flowchart of Fig. 12. Fig. 12 Flowchart of ray synthesis.

CONCLUSION
Ultra-wide area FTV with full parallax is realized by 4D orthogonal ray-space processing. Ray-space of "a group of rays through one point" is derived in 4D orthogonal ray-space. It is used for ray capture, view generation and ray synthesis. Ray-space of "a group of rays through one point" forms a hyperbolic curve in (x, y, tanθ) space with fixed φ and a straight line in (x, y, tanφ) space with fixed θ. By varying the camera position along a line, ray-space captured by linear arrangement cameras is obtained. This ray-space forms a hyperbolic cylinder in (x, y, tanθ) space with fixed φ and a plane in (x, y, tanφ) space with fixed θ.
Rays for view generation of ultra-wide area FTV are synthesized by interpolating the captured ray-space using the concept of "a group of rays through one point". It is done so that the intersections of the captured ray-space and the ray-space of rays emitted from a light source have the same color.