D-Lib Magazine
February 2002

Volume 8 Number 2

ISSN 1082-9873

Digital Collections of Real World Objects

Hendrik P.A. Lensch, <[email protected]>
Michael Goesele, <[email protected] >
Hans-Peter Seidel, <[email protected]>
Max-Planck-Institut für Informatik
Saarbrücken, Germany

Abstract

Real world objects, such as works of art, archeological artifacts and even common everyday objects, exhibit large variations in color due to the way light is reflected from their surfaces. A high quality digitization method must be capable of capturing these effects if the digital models generated from the real objects are to look realistic.

In this article, we present an efficient method for acquiring high quality models of real world objects. The resulting digital models can be viewed under arbitrary viewing and lighting conditions. The efficient acquisition technique, small size, high quality, and versatility of the generated models make this technique well suited for large digital collections.

Introduction

Integrating 3D objects into digital documents is a challenging task for two reasons: A great deal of effort is required to create highly detailed 3D models and their inherent complexity makes storage, transmission, and interactive display difficult. Nevertheless, there are many digital documents for which highly detailed 3D representations of real world objects are needed, such as advanced e-commerce applications or multimedia databases like digital libraries, online encyclopedias or virtual museums. The success of these applications strongly depends on advances in the field of high quality object acquisition, representation, distribution and rendering.

In this article, we focus on the acquisition of faithful, highly detailed representations of real world objects through the use of an efficient, image-based technique. The resulting representations will contain both the objects' geometry and the appearance of the surfaces, i.e., their reflection properties. A more in-depth description of the techniques described below can be found in [10], and an overview of the entire field of high quality object acquisition, representation, distribution and rendering can be found in [9].

Describing Surface Attributes

The appearance of an object — its "look" — is determined by the properties of its surface such as the (diffuse) color. However, a single color value or even multiple color values (in a texture) are not sufficient to fully describe a real world object. For example, the difference between a matte and a glossy surface cannot be expressed as a color value. Several metrics such as gloss or haze are used to describe individual appearance properties of an object. (See [5] for an overview.)

A more general measure is the bi-directional reflectance distribution function (BRDF) that describes how light is reflected at the surface of an object. More formally, a BRDF is a four-dimensional function that describes which portion of the light hitting a surface from an incident direction is reflected into an outgoing direction. The incident light is scattered at the surface and distributed in many directions (see Figure 1).

Figure 1: Illustration of a BRDF: Light is hitting the surface at point x from an incoming direction (t)_i and is reflected into direction (t)₀. All directions are given relative to the surface normal n. If (t)₀ is varied while (t)_i is kept constant, the BRDF describes the amount of light reflected in direction of (t)₀ . Highlights are caused by the specular part of the BRDF where light is reflected mainly around the mirror direction. The small, spherical part corresponds to diffuse reflection where light is equally distributed in all directions.

All of these metrics are useful for describing the surface of an ideal object made of a single homogeneous material. However, most objects encountered in the real world consist of several different materials. They are almost never perfect but instead show small imperfections like material variations, scratches, or accumulated dirt. This is especially true for works of art or archeological objects, which are of special interest for digital collections. A very precise way to represent these details is to assign a different BRDF to each surface point that leads to a spatially varying BRDF. Without these details, objects tend to look artificial and unrealistic (see Figure 2).

Figure 2: Comparison between an object rendered with five different BRDFs (one for each basic material) and a spatially varying BRDF. The added details help to make the object look more realistic.

Previous Work

Several methods for the acquisition of surface properties have been described in the literature. Some of these methods have focused on homogeneous materials [17, 7, 12, 13] neglecting spatially varying properties of objects. Other methods assume that only the diffuse part of the BRDF (i.e. the color of the object) varies over the surface while the specular part (reflections around the mirror direction — see Figure 1) remains fixed [19, 14, 15] - an assumption that does not hold for many practical cases.

A very general approach has been proposed by Debevec et al. [1], who have measured spatially varying BRDFs without making any additional assumptions. However, this method requires several hundred input images for a single point of view and requires a huge effort for the acquisition and storage of the resulting models.

In our work, we concentrate on measuring spatially varying BRDFs for the entire surface of an object using only a small number of high dynamic range photographs [2] (about 15-25 images), thereby speeding up the acquisition phase significantly. In particular, our contributions are:

• A robust and efficient BRDF fitting process that clusters the acquired samples into groups of similar materials and fits a Lafortune BRDF model [7] to each group,

• A method that projects the collection of samples of each surface point into a basis of BRDFs obtained from the clustering procedure. This projection accurately represents the material, at that point, and results in a compact representation of a truly spatially varying BRDF.

As a result, we obtain a compact representation of spatially varying materials. The method works both for objects consisting of a mixture of distinct materials and for smooth transitions between material properties.

Data Acquisition

For our measurements, we acquire the object's geometry with a standard structured light 3D scanner or other 3D scanning devices. In order to increase quality and to reduce memory consumption, the resulting triangle mesh is smoothed, manually cleaned, and decimated. It can even be transformed into a level-of-detail representation for faster transmission. In this article, we focus on the acquisition of surface attributes. A detailed overview over mesh acquisition and processing techniques can be found in [6].

Surface attributes are captured in a second step using an image-based technique. We capture all images using a professional-level digital camera after calibrating the camera�s intrinsic parameters [20], in order to have a known relationship between pixels in an image and points in space. The BRDF measurements are performed in a lab covered with dark felt [4] to reduce the influence of the surroundings on the measurements as much as possible. A special light bulb of known brightness serves as point light source for the BRDF measurements.

Several views of each object are captured with different camera and light source positions. For each view, we acquire a series of photographs of the object lit by the point light source, with varying exposure time from which a high dynamic range image [2] is calculated. After calibrating with the known brightness of the lamp, each pixel of the high dynamic range images contains full range, floating point radiance samples. Furthermore, we take two images to recover the light source position relative to the camera and one image of the object's silhouette to register the 3D geometry model with the images [8]. In a production environment, the 3D scanner, camera, light source, and the test object could be combined into single, calibrated gantry to speed up the acquisition process and to render the registration unnecessary.

Figure 3: A view of the acquisition setup in the photo studio with light source, camera, calibration target for the light source position, and test object.

Data Preprocessing

In the acquisition phase, we collected several different types of data such as a polygonal mesh describing the geometry of the object and the reflectance samples from the images. After acquisition and registration of geometry and image data, it is necessary to merge and rearrange the data for further processing. For each point on the model's surface, we collect all available information in a data structure called a lumitexel. It contains the following information:

• the position of the surface point (its coordinates)

• the orientation of the surface at the surface point (represented by the surface normal)

• the photometric data for each of our input images in which the surface point was illuminated by the light source and visible from the camera�s position. This data includes the direction to the light source and the camera as well as the amount of light that is reflected into the camera.

BRDF Generation

From the collected data we have to generate a BRDF for each surface point capturing its reflection properties. The process of BRDF generation, which can be broken down in BRDF fitting, clustering of lumitexels, and projection, is described in general below and in more detail in Appendix 1.

Actually, a lumitexel can already be seen as a very sparsely sampled BRDF in a tabulated representation, typically with only four to ten entries. But instead of using the radiance samples captured in the lumitexel directly, we will represent the surface appearance by a mathematical BRDF model whose parameters have to be estimated with respect to the error between a lumitexel and the BRDF.

This representation mainly has two advantages: first, only the parameters of the BRDF model have to be stored instead of a list of radiance samples; and secondly, a BRDF model is defined even for incident and outgoing directions that have not been acquired, providing smooth data extrapolation.

The number of radiance samples per lumitexels is too small to obtain faithful BRDF parameters from a single lumitexel. However, the parameters can be estimated accurately for a whole group or cluster of lumitexels, i.e. by increasing the number of samples. The given lumitexels are therefore partitioned into clusters so that each cluster corresponds to just one basic material of the object. The general idea of the clustering is to first fit a BRDF to an initial cluster consisting of all lumitexels. Then we generate two new BRDF models representing two new clusters. The lumitexels from the original cluster are then distributed according to their distance to the generated BRDFs into the new clusters. New BRDF models are then fitted to the two clusters which best approximate the lumitexels in the new clusters. To obtain a clear separation between the generated clusters, we repeat the steps of distributing the lumitexels and BRDF fitting until the clusters are stable.

As can be seen in Figure 2, above, the representation of an object by a collection of only a few clusters and corresponding BRDFs make the virtual object look artificial because real surfaces exhibit changes in the reflective properties, even within a single material. These changes cannot be represented by a single BRDF per cluster since all lumitexels within the cluster would be assigned the same BRDF parameters. To obtain truly spatially varying BRDFs, we had to find a specific BRDF for each lumitexel. See Appendix 1 for a more detailed description of the projection process.

Results and Conclusions

In this article, we present a method to generate high quality 3D models of objects including their reflection properties for each surface point. Compared to other methods that capture only the colors of an object (e.g., in a standard texture), the spatially varying reflection properties increase the realism to a great extent. Compared to other approaches that capture reflection properties such as surface light fields or reflectance fields [18, 1], this method requires only a small acquisition effort and leads to a very compact representation of the resulting 3D models, which can be viewed under arbitrary viewing and lighting conditions.

The following figures present some models acquired with our method. In addition, Appendix 2 provides links to movies that show our models under varying viewing and lighting conditions. The model of the clay bird (see Figure 2) illustrates the importance of spatially varying reflection properties. The bronze bust in Figure 4 below shows another reconstructed object with very different reflection properties. The bronze look is very well captured.

Figure 4: Model of a bronze bust. Note that it looks realistic even under modified lighting conditions.

Figure 5 compares an object rendered with an acquired BRDF and a photograph of the object. There are only a few differences in the highlights because an inadequate number of radiance samples were captured. Capturing more samples or images will increase the quality of the object model.

Figure 5: Comparison between a photograph (left) and a rendered image of a 3D model under similar lighting conditions.

Application Areas

Apart from generating a more accurate and visually appealing representation of an object, the method described in this article has several desirable properties that open up new possibilities for digital collections of real world objects:

• The method requires only a small number of input images, speeding up the acquisition process. Although our current research prototype requires manual intervention during the acquisition process, a robot-controlled gantry could lead to an almost fully automatic system.

• The relatively small size of the resulting models is ideal for environments such as those digital libraries that have limited storage capacity or limited bandwidth.

• The high quality of the objects and the ability to view them under arbitrary viewing and lighting conditions make them useful for a wide range of applications in entertainment, edutainment, and scientific research. Virtual collections of art works originally located at different sites all around the world can be built and presented on-site or (using simplified versions of the models) via the Internet.

Future Work

In the future, we plan to extend the class of objects that can be captured with our method (e.g., to objects with anisotropic surfaces such as brushed metal). We would also like to further increase the accuracy of the results and to simplify the acquisition process by making it more and more automatic.

Appendix 1: Principle of BRDF Fitting

Appendix 2: Movies Illustrating the Results of the Method

Bibliography

Top | Contents
Search | Author Index | Title Index | Back Issues
Previous article | Next article
Home | E-mail the Editor

D-Lib Magazine Access Terms and Conditions

DOI: 10.1045/february2002-goesele