Discover Cinematic Rendering technology
November 23, 2015
by Gus Iversen, Editor in Chief
Q&A with Dr. Dorin Comaniciu Vice President of Siemens’ Medical Imaging Technologies
The images you’re looking at on the following pages may look like something out of a video game or science fiction movie, but there’s nothing fictional about them. These are highly detailed renderings of actual living individuals — and they were produced using Siemens’ cinematic rendering technology.
To see a portfolio of dramatic cinematic rendering images, courtesy of Siemens Healthcare, please click here
These stunning depictions represent the cutting edge of data visualization, and HealthCare Business News spoke with Dr. Dorin Comaniciu, vice president of Siemens’ Medical Imaging Technologies in Princeton, New Jersey, about how this technology came to be — and what role it may play in the future of medical imaging.
HCB News: What is cinematic rendering technology?
Dr. Dorin Comaniciu: Today medical imaging scanners, like CT and MR, deliver very detailed data representations of the interior of the human body. Cinematic rendering is an innovative algorithm that addresses the visualization of such data in order to provide superb photorealistic images. The inspiration for our scientists was the film animation industry: design a new rendering technology that helps to create visualizations that are increasingly realistic. We believe such technology opens new possibilities for medical education, early disease detection, planning and guiding of surgical procedures, and ultimately facilitates better communication between referring physicians and patients.
HCB News: How was it developed?
DC: Our scientists asked the question: how can we advance the visualization of medical data? The images provided today by MR and CT have incredible resolution and so many details that are useful to the physician. However, sometimes they are difficult to visualize. To understand the complexity of this data, imagine a large-volume CT or MR of the whole body. You can think of a cube made up of several hundred million voxels (i.e., elements of the 3D Cartesian grid), each voxel characterizing specific information about the structure within the human body, such as density or tissue type. How do you visualize such a large amount of voxels? How do you make sure you properly see the rich content present in this data? Assuming light in the surrounding environment can reach any location from a myriad different directions, the main idea is to model the light propagation — the interaction of the light rays with the data — and to compute and display the resulting properties of the rays that reach the human observer.The integral equation that models the underlying geometric optics is quite complex.However, it can be solved iteratively through a process known as Monte Carlo integration. This is essentially what we implement through cinematic rendering: To reveal the volumetric information of interest, we iteratively propagate billions of light rays through the original volume, and due to parallelization and optimization algorithms such as importance sampling, this process runs in, close to, real time. In this way, by modeling the light scattering and the bouncing of rays, one can accurately represent the content of volumetric data. We can observe soft shadows, depth of field, subsurface scattering, caustics, motion blur… everything that makes the images very realistic.
HCB News: What are the potential applications?
DC: This new technology has the potential to enhance clinician-patient communication. Instead of seeing traditional black and white images, if you look at Image 1 of our presentation you see a great visualization of the neck, including the carotid artery, jugular vein and thyroid gland. Suddenly, things are much simpler to understand compared to a traditional CT or MR image.
Second, it’s about revealing more human anatomy and function through medical imaging. During discussions with our key luminaries, we talked about a turning point in teaching anatomy. Today most of academic teaching is done on cadavers through dissection, but you don’t deal with the living body. The fascinating part here is thinking of the opportunity to teach about the living body via a virtual dissection facilitated by medical imaging and photorealistic visualization. Third, we focus on increasing the sensitivity of diagnostic imaging. Cinematic rendering can contribute to revealing disease in early stages.
Think, for example, about enhancing the conspicuity of lung nodules in CT data. We provide an example of nodules in Image 2. Here, through the capturing of detailed image properties, you can see a much richer depth representation of data. Furthermore, we can focus on better staging, and better quantification of tumors. In general, photorealistic visualization brings value every time one needs to quantify pathologies in images. The fourth impact is about decision-making. Can we increase specificity? What is the type of disease? Can we better stratify or classify disease? Can we derive more information from the data? What is the status of implants?
Multiple coronary stents are highlighted in Image 3. Then, No. 5, if one sees more through medical imaging, one can better guide minimally invasive procedures such as aortic stent implants, heart valve implants, and heart, liver or spine surgery. Minimally invasive surgery is developing rapidly today to minimize trauma on the patient and reduce the associated cost. Medical imaging constitutes the main source of information for guiding such procedures.
HCB News: What’s next for this technology?
DC: Personalized therapy is the next big topic, and we are addressing it on the research side. We think of obtaining information not only at the anatomical level, but also at the physiological one. Take the heart, for example. You have the heart anatomy together with information about electrical signal propagation, the dynamics of blood being pumped by the heart, and the electromechanical information on how the heart muscles function. All of this information comes together to form a comprehensive, virtual, yet personalized representation of the patient’s heart. Such models can become the base for future individualized therapies.
One last comment: all images that we present in this article are images of real, living patients. When you look at the brain, it is a living brain. The image has been created through MR, noninvasively, and yet it looks so realistic. We have much more to explore to understand the full potential of this fascinating technique of cinematic rendering. Siemens Healthcare plans to display new cinematic rendering images at its exhibit hall booth (booth #4136, South building of Mc-Cormick Place, Hall A) during RSNA.
The images you’re looking at on the following pages may look like something out of a video game or science fiction movie, but there’s nothing fictional about them. These are highly detailed renderings of actual living individuals — and they were produced using Siemens’ cinematic rendering technology.
To see a portfolio of dramatic cinematic rendering images, courtesy of Siemens Healthcare, please click here
These stunning depictions represent the cutting edge of data visualization, and HealthCare Business News spoke with Dr. Dorin Comaniciu, vice president of Siemens’ Medical Imaging Technologies in Princeton, New Jersey, about how this technology came to be — and what role it may play in the future of medical imaging.
HCB News: What is cinematic rendering technology?
Dr. Dorin Comaniciu: Today medical imaging scanners, like CT and MR, deliver very detailed data representations of the interior of the human body. Cinematic rendering is an innovative algorithm that addresses the visualization of such data in order to provide superb photorealistic images. The inspiration for our scientists was the film animation industry: design a new rendering technology that helps to create visualizations that are increasingly realistic. We believe such technology opens new possibilities for medical education, early disease detection, planning and guiding of surgical procedures, and ultimately facilitates better communication between referring physicians and patients.
HCB News: How was it developed?
DC: Our scientists asked the question: how can we advance the visualization of medical data? The images provided today by MR and CT have incredible resolution and so many details that are useful to the physician. However, sometimes they are difficult to visualize. To understand the complexity of this data, imagine a large-volume CT or MR of the whole body. You can think of a cube made up of several hundred million voxels (i.e., elements of the 3D Cartesian grid), each voxel characterizing specific information about the structure within the human body, such as density or tissue type. How do you visualize such a large amount of voxels? How do you make sure you properly see the rich content present in this data? Assuming light in the surrounding environment can reach any location from a myriad different directions, the main idea is to model the light propagation — the interaction of the light rays with the data — and to compute and display the resulting properties of the rays that reach the human observer.The integral equation that models the underlying geometric optics is quite complex.However, it can be solved iteratively through a process known as Monte Carlo integration. This is essentially what we implement through cinematic rendering: To reveal the volumetric information of interest, we iteratively propagate billions of light rays through the original volume, and due to parallelization and optimization algorithms such as importance sampling, this process runs in, close to, real time. In this way, by modeling the light scattering and the bouncing of rays, one can accurately represent the content of volumetric data. We can observe soft shadows, depth of field, subsurface scattering, caustics, motion blur… everything that makes the images very realistic.
HCB News: What are the potential applications?
DC: This new technology has the potential to enhance clinician-patient communication. Instead of seeing traditional black and white images, if you look at Image 1 of our presentation you see a great visualization of the neck, including the carotid artery, jugular vein and thyroid gland. Suddenly, things are much simpler to understand compared to a traditional CT or MR image.
Second, it’s about revealing more human anatomy and function through medical imaging. During discussions with our key luminaries, we talked about a turning point in teaching anatomy. Today most of academic teaching is done on cadavers through dissection, but you don’t deal with the living body. The fascinating part here is thinking of the opportunity to teach about the living body via a virtual dissection facilitated by medical imaging and photorealistic visualization. Third, we focus on increasing the sensitivity of diagnostic imaging. Cinematic rendering can contribute to revealing disease in early stages.
Think, for example, about enhancing the conspicuity of lung nodules in CT data. We provide an example of nodules in Image 2. Here, through the capturing of detailed image properties, you can see a much richer depth representation of data. Furthermore, we can focus on better staging, and better quantification of tumors. In general, photorealistic visualization brings value every time one needs to quantify pathologies in images. The fourth impact is about decision-making. Can we increase specificity? What is the type of disease? Can we better stratify or classify disease? Can we derive more information from the data? What is the status of implants?
Multiple coronary stents are highlighted in Image 3. Then, No. 5, if one sees more through medical imaging, one can better guide minimally invasive procedures such as aortic stent implants, heart valve implants, and heart, liver or spine surgery. Minimally invasive surgery is developing rapidly today to minimize trauma on the patient and reduce the associated cost. Medical imaging constitutes the main source of information for guiding such procedures.
HCB News: What’s next for this technology?
DC: Personalized therapy is the next big topic, and we are addressing it on the research side. We think of obtaining information not only at the anatomical level, but also at the physiological one. Take the heart, for example. You have the heart anatomy together with information about electrical signal propagation, the dynamics of blood being pumped by the heart, and the electromechanical information on how the heart muscles function. All of this information comes together to form a comprehensive, virtual, yet personalized representation of the patient’s heart. Such models can become the base for future individualized therapies.
One last comment: all images that we present in this article are images of real, living patients. When you look at the brain, it is a living brain. The image has been created through MR, noninvasively, and yet it looks so realistic. We have much more to explore to understand the full potential of this fascinating technique of cinematic rendering. Siemens Healthcare plans to display new cinematic rendering images at its exhibit hall booth (booth #4136, South building of Mc-Cormick Place, Hall A) during RSNA.
