Embed Size (px)
Citation preview
PHOTOREALISTIC RENDERING OF SCIENTIFIC DATA
Joel Ogden1, Jabari Jordan1, Chelsey Krol2, Tanya Papazian3, Hans-Peter Bischof1, Reynold Bailey1
1Department of Computer Science, Rochester Institute of Technology, Rochester, United States
2Department of Computer Science, Mount Holyoke College, South Hadley, United States
3Department of Computer Science, Montclair State University, Montclair, United States
[email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Keywords: Scientific Visualization, Maya, Spiegel
Abstract: Generating photorealistic images of astrophysical simulations can enhance the experience of watching galac-tic visualizations for both the specialists who study the data and the average person who is simply interestedin outer space. Unfortunately, the astrophysicist who is creating the simulations typically lacks the expertiserequired to generate photorealistic images. Likewise, a 3D artist may be unaware of the physics behind cer-tain astrophysical events. We aim to use Spiegel, a user interface that controls the rendering of astrophysicaldata and Maya, a high end 3D animation program, to allow a non-artist to easily create renders of photore-alistic images. Spiegel provides a user-friendly interface for controlling the creation of potentially complexrendering applications by individuals with little experience in computer programming. Since Spiegel’s basicvisualization capabilities are limited to simple primitives like points and lines, it was necessary to develop anadditional program for Spiegel to interface with Maya’s 3D rendering capabilities. This software interface iscalled Miegel. Using Miegel, the astrophysicist now has access to Maya’s 3D rendering capabilities allowingthem to create stunning visualizations of astrophysical phenomena. In addition, new artistic effects can becreated with Maya in the form of presets, which can be integrated into the user’s visualization with minimalknowledge of computer programming.
1 INTRODUCTION
Visualizations of scientific processes have alwaysbeen a crucial part of education and understandingto the student and scientist alike. Being able to seesomething as opposed to reading it from a textbookallows for a multi-dimensional learning experienceand a more dynamic understanding of complex sub-ject matter. Unfortunately not all visualizations are aseffective as they could be. This is because most sci-entists do not have the training or skills in modern 3Drendering techniques to produce compelling visuals.Spiegel (Bischof et al., 2006) was developed to bridgethis gap by bringing tool that were previously onlyavailable to a 3D artist into the hands of the scientist.Spiegel is a software package that allows individualswith little or no 3D rendering or programming ex-perience to create simulations of scientific processesbased on data. The program in its basic form is how-ever limited to how the data is displayed. Most visu-alizations are displayed as simple points, which takeon the roles of stars or particles. Figure 2 shows an
Figure 1: Photorealistic rendering of a black-hole mergercreated using our Spiegel-Maya interface called Miegel.
example rendering created using Spiegel’s basic ren-dering capabilities. We created Miegel, a software in-terface that allows Spiegel to take advantage of therendering and animation capabilities of Maya. By us-ing our system, the user is able to create more realisticand appealing visualizations such as the one shown inFigure 1.
Figure 2: A visualization of a black-hole merger createdusing Spiegel’s basic rendering capabilities. The stars aredisplayed as 3D points.
Figure 3: Overview of the Spiegel framework. Data is ex-tracted from one or more file servers and distributed to acluster of computers. Each processing unit in the clustergenerates one (or more) frames of the complete visualiza-tion sequence in parallel. These frames are then combinedto create a video. Image courtesy of (Espinal et al., 2010).
2 SPIEGEL
Spiegel is a software package that allows users withlittle experience in 3D digital graphics or computerprogramming to create visualizations based on rawsimulation data. Spiegel makes this possible througha graphical user interface that allows users to com-bine components to create their desired visualization.Each component represents an individual program de-signed to perform a specific task. The Spiegel systemis designed to be extensible by allowing new compo-nents and functionality to be added to the basic frame-work. This is possible because Spiegel is designedusing a data flow architecture. Components havecommunication endpoints, which can be connectedto form a complete visualization program. When theprogram is executed, data is passed from one compo-nent to another. Each component performs specific
Figure 4: Example of a program created in Spiegel that il-lustrates the data flow architecture. Image courtesy of (Es-pinal et al., 2010).
operations that contribute to the final result (Espinalet al., 2010). Communication between componentsis made possible by a variety of built-in data struc-tures (Bischof and Dong, 2011). Data is taken in viaan extractor and placed in these structures which arethen passed between components as optional param-eters. Figure 4 shows a simple visualization programcreated using Spiegel’s graphical user interface.
3 PREVIOUS WORK
Spiegel’s basic rendering capabilities are provided us-ing OpenGL primitives such as points and lines. Ascan be seen in Figure 2, the resulting renderings areuseful but not necessarily compelling or realistic.
Several efforts have been undertaken to improvethe quality of the renderings produced by the Spiegelsystem. One such effort expanded the visualizationcapabilities of Spiegel by using Pixar’s PhotoRealis-tic RenderMan R� (Cassidy et al., 2011). By using theshader library accessible through the RenderMan R�
program, the researchers were able to apply displace-ment shaders to better represent stars and other cos-mic forms. They also used noise patterns and colorpalettes to develop realistic models of stars. A fur-ther advantage of RenderMan R� was its ability to in-corporate realistic lighting effects, which proved toproduced far more compelling visualizations.
In addition, other work has been done to studythe gaze patterns of expert and novice astrophysiciststo gain insight into what makes a favorable or com-pelling visualization (Arita et al., 2011). By hav-ing participants view a number of images of galacticevents, both photographed and computer generated,the researchers were able to gain a better understand-ing of preferred camera positions and zoom settings.This information was then used to modify existingSpiegel visualization programs to create more favor-able renderings.
Despite these advances, the improved Spiegelframework had several major drawbacks. In partic-ular, developing custom shaders for the RenderMan R�
component required significant programming exper-tise. Furthermore, the system still lacked the capa-bility to handle complex animations and fluid simula-tions.
To overcome these limitations, we decided to useMaya because of its ability to produce compellingvisualizations with minimal programming. Further-more, Maya has a robust fluid simulation systemand the ability to store artist created presets, whichnovices can easily access. This fluid system provedto be the ideal solution for simulating space dust andgases.
4 MAYA
Maya is a high-end 3D animation program capa-ble of producing stunning visual graphics. A majorshortcoming is that it takes a considerable amount oftime and training in order for an individual to pro-duce these images. Through the Miegel (i.e. Maya-Spiegel) interface that we created, the user can takepreset files created by an expert in the use of Mayaand insert them into the scene with relative ease. Us-ing the controls provided in the Miegel interface, theuser can also customize the visualization by simplyadjusting various attributes.
In order to produce the visual effects of spacedust and gases, we utilized Maya’s fluid simulationsystem. Through the careful application of noise at-tributes and color gradients we were able to constructcloud shapes that were much like those seen in galac-tic imagery. These settings were then saved as presets,which allow for faster render times and greater con-trol. These types of presets can easily be loaded andadjusted by a novice user to quickly create appealingvisualizations.
Another powerful feature that Maya provides isthe ability to map an image onto a fluid. This is doneby using the color values of the image to representspecific densities in the fluid. Maya’s fluid simulatorcan then use this information to produce the effect ofvortices and cloud formations (Brinsmead, 2007). Wewere able to add further detail and realism to the fluidsimulation through the use of photographs of variouscloud formations made available through the NationalOceanic and Atmospheric Administration, as well asother open source venues. This same technique can beapplied for any photographed astrophysical phenom-ena, allowing for an added degree of dimensionalityto the visualization. By utilized Maya’s wide array ofphysically accurate modifiers, we were able to manip-ulate the fluid as if it were acted upon by forces foundin space.
5 MIEGEL
Miegel is a software interface designed to integrateMaya’s rendering capabilities with Spiegel. Miegelwas created because the existing solutions providedby OpenGL and RenderMan R� were too complicatedfor novices to use since they required significant pro-gramming experience. The process of adding newtypes of effects to the visualization pipeline, for ex-ample, required many hours of implementation, test-ing, and debugging. Miegel accomplished this by us-ing the options available through Maya to producestunning visualizations, which in tern require less pro-gramming hours. In addition, Miegel aimed to allowfor visuals produced by a 3D graphics artist to be eas-ily integrated into the visualization. With Miegel, thetask of creating visual effects such as those seen inprofessional productions as well as the process of in-tegrating them into the visualization is greatly sim-plified. This is done by the user interface in Miegel,which allows for visuals produced by a 3D graphicsartist in Maya to be easily integrated into the visual-ization through a drop-down menu. The applicableattributes of the specific preset are then displayed forthe user to modify, giving the user full control overthe aesthetics of the finished rendered image.
Miegel works by producing a MEL (Maya Em-bedded Language) script and a data cache PDC (Parti-cle Disk Cache) and passing them to Maya. The MELscript contains information about the objects Mayashould create, such as color and luminance values forthe given particles, as well as scripts to import addi-tional scene files. The PDC is a raw file format usedby Maya as a way of caching information about theparticles. The cache maps attribute information aboutparticles to disk allowing for faster drawing of par-ticles, versus having to re-calculate the positions foreach particle for each frame (Center, 2005). The PDCcontains the vector data for the particles that repre-sent the stars in the simulation. Maya can then ren-der this out into a batch of image files, which can beat full HD resolution (in contrast with the previoussolution which could only produce standard resolu-tion images). The big advantage with this system isthat Miegel automates most of the process and allowsusers unfamiliar with Maya to still take advantage ofits powerful capabilities.
Figure 5 illustrates the Miegel architecture. AMEL script is a language provided by Maya thatallows users to create macro programs to automatemany of Maya’s functions and operations. Becausethe Maya GUI is itself written in MEL, any opera-tion that can be performed by a human using Mayacan also be programmed in MEL. This means that
a program like Miegel that generates a MEL scriptcan perform any action in Maya that is available to auser. The simplicity of MEL scripts mean that Miegelcan easily be expanded to utilize any of Maya’s fea-tures (Autodesk, 2010b).An example of a MEL script is
particle n star p 0 0 0;
setAttr star.prt 8;
setAttr particleCloud1.incandescence
type double3 1 1 1;
setAttr particleCloud1.glowIntensity 1;
These few lines instruct Maya to place a singleparticle named star at vector (0, 0, 0), convert the typeof the particle to a cloud, then change the incandes-cence color to white with maximum glow. Once thissingle particle is created, it can be attached to the PDCthat contains all of the particles (potentially tens ofthousands) and an image that matches the data in thesimulation can be rendered. Scripts such as the oneshown above are automatically generated by Miegeland utilized by Maya to produce the resulting image.
Spiegel utilizes simulation files with the exten-sion .sim. A sim file is a human readable file writ-ten in plain text, which contains information such asthe number of stars in the simulation, the positionsof the stars for each frame, as well as their velocity,acceleration and other information. Miegel creates adisk cache first be parsing out information from thesim file. For the purpose of the visualization, we onlycared for the positions and frame number of the stars.To convert the .sim to PDC, Miegel makes two passesover the .sim file. First, the positions of the stars andtheir frame number are extracted, at the same timefilling any gaps in the data. This information is thenwritten as raw bytes into an .msf (Maya-Spiegel For-mat) file. From here, Miegel can quickly extract thedata for any particular star in file from any frame. Thebenefits of having the .msf file is that the .sim fileonly has to be parsed once and any subsequent vi-sualizations using the same data does not have to berun through the converter again. For reference, a 7GB.sim file took over an hour to be convert to msf. Otherbenefits of the .msf format include a much smallerfile size and predictable data placement allowing forclose to real-time access of information, this allowsfor frames deep in the simulation to be quickly ex-tracted and used for the visualization.
After the MEL script is generated and the PDCcompleted, Maya can read the script and output aMaya scene file (.ma) which holds all the informationneeded by the renderer. The .ma file is the standardfile format used by Maya that can be read and edited,allowing Maya users to verify the visualization before
Miegel
Visualization Attributes
Simulation Data Interpreter
Maya Scene Files
Maya 3D
Maya Renderer
MEL Script Generator
Particle Disk Cache Creator
Maya Scene File
Spiegel
Visualization Settings
.sim to .msf Convertor
Maya Effects Importer
Figure 5: Architecture of Miegel. The Simulation Data In-terpreter takes the position data of the stars, and creates theParticle Disk Cache, which is needed for Maya to map thelocation of the stars to the particles objects. The visual-ization attribute module takes in information from the userabout the desired appearance of the stars (including color,size, and glow intensity) and produces a MEL script. Thescript can also include instructions to integrate other Mayafiles into the visualization. Once the script is generated, itis executed by Maya to produce the individual frames of thevisualization
rendering and import other Maya scene files. The .mafile is read by the Maya renderer to produce the finalimages. The render can be flagged with arguments tochange the attributes of the outputted file, such as res-olution, and what rendering techniques it should use,such as anti-aliasing. This provides other advantagesover the OpenGL and RenderMan R� solutions, wheresuch features must be explicitly programmed.
Spiegel components were created to allow for easyaccess to Miegel from Spiegel. A user familiar withSpiegel can easily use the new Miegel component toaccess Maya. The user can import the required simu-lation file, then choose which frames in the simulationto render. Furthermore, they can import as many 3rd
Simulation.msf3
Extractor
Clock
Miegel
Star Attributes (Colors, size, etc.)
Space Cloud Effects.ma
Background Stars.ma
Maya Renderer
Start, Stop, FPS
Rendering Settings
Maya File Importer
Figure 6: : The Spiegel components available to the userand how they interact to produce visualization.
party Maya scenes that are available, having access toeasy to use sliders and drop down boxes to choose at-tributes, which change the look of the stars as well asthe image output format, location, resolution, as wellas other renderer attributes. Figure 6 illustrates howthe information flows through Spiegel into Miegel andthe renderer.
Figure 7: Maya viewport view of the simulation.
6 CONCLUSIONS AND FUTUREWORK
We have created a highly customizable and robustprogram called Miegel that is able to take simulationdata in its raw form from Spiegel and send it to Mayato generate compelling visualizations. This allows theuser to utilize the full gambit of Maya’s capabilitiesin Spiegel in order to further enhance their visual-ization. With Miegel, Spiegel could be expanded touse professional grade animation programs to betterits ability to create realistic and eye catching simula-tions. This provides the ability to add numerous spe-
cial effects and/or programs to Spiegel in the future.In future work it would be possible for the user to usethe integrated camera capabilities, allowing for nearlyseamless incorporation of stereoscopic cinematogra-phy into scientific simulations. Maya also has soni-fication capabilities, allowing the user to coordinatesound with the aesthetics of the visualization, whichwould greatly enhance the viewing experience. Mayaalso allows the use of it’s diverse library of shadersand the ability to create customizable shaders, givingeven greater customization to the aesthetics of scene.In addition, Maya’s newest release has the capabilityto render images using a GPU based rendering systemthat has the capability to display these images in theviewport in their rendered state. Figure 7 shows theviewport view in Maya of a simulation that was gener-ated from our program Miegel. It is entirely feasibleto integrate other animation and rendering programsinto Spiegel, which will better enhance the simula-tions that are produced. The framework we have cho-sen allows for the addition of any preset the user mayrequire, giving the program an endless potential forexpansion that is only limited to one’s imagination.
ACKNOWLEDGEMENTS
This material is based upon work supported bythe National Science Foundation under Award No.0851743. Any opinions, findings, and conclusions orrecommendations expressed in this material are thoseof the authors and not necessarily reflect the views ofthe National Science Foundation.
REFERENCESArita, J., Feliz, J., Rodriguez, D, B. H., Rege, M., and Bai-
ley, R. (2011). Creating audiencespecific galactic sim-ulations using eye-tracking technology. In in proceed-
ings of International Conference on Information Visu-
alization Theory and Applications.Autodesk, I. (2010a). Autodesk maya help.Autodesk, I. (2010b). Maya api reference.Bischof, H.-P., Dale, E., and Peterson, T. (2006). Spiegel
- a visualization framework for large and small scalesystems. In Arabnia, H. R., editor, Proceedings of
the 2006 International Conference on Modeling, Sim-
ulation and Visualization Methods, pages 199–205.CSREA Press.
Bischof, H.-P. and Dong, A. (2011). Directing a visual-ization ala kubrick. In Proceedings of The 2011 In-
ternational Conference on Modeling, Simulation and
Visualization Methods.Brinsmead, D. (2007). Volume displacement method for
clouds using fluids.
Cassidy, P., Kilburn, T., Salemink, V., Bailey, R., andBischof, H.-P. (2011). Improving the visualization ofgalactic events using pixars renderman. In Poster Pa-
pers Proceedings of 19th International Conference on
Computer Graphics, Visualization, and Computer Vi-
sion, 2011.Center, S. D. S. (2005). Maya pdc format simplified.Espinal, J., Allen, V., Amable, K., Bailey, R., and Bischof,
H.-P. (2010). Renderman’s power to visualization’srescue. In Communication Papers Proceedings of
18th International Conference on Computer Graph-
ics, pages 243–249.
