SOFTWARE DEVELOPER’S QUARTERLY Issue 16• Jan 2011
Editor’s Note...... 1 PARAVIEW 3.10 The ParaView team is gearing up for the next official release of ParaView, 3.10. Once released, the binaries will be avail- Recent Releases ...... 1 able for download on the ParaView download page: http:// paraview.org/paraview/resources/software.html. This release Medical Image Analysis with ITK on Apple iOS ...... 3 features notable developments, including mechanisms to incorporate advanced rendering techniques, improved Ultrasound and ITKv4 ...... 6 support for readers and several usability enhancements and bug fixes. New Functionalities for Spherical Demons Registration..... 8 For the 3.10 release, we have refactored and vastly improved Adapting ITK Framework to Fit Parametric the VisIt-bridge plugin and several new file formats such as Image Models ...... 10 Silo, CGNS and Tecplot are now supported. Developers can also create ParaView reader plugins for VisIt readers. A Lightweight Image Comparison Library ...... 13 With 3.10, we have included a Python-based calculator which makes it possible to write operations using Python. Using Kitware News ...... 15 the Python calculator, it is possible to use advanced functions such as gradients, curls and divergence easily in expression. There is also a new Eyedome Lighting (EDL), a non-photore- alistic shading technique designed at EDF (France) to improve depth perception in scientific visualization images (Figure 1). It relies on efficient post-processing passes implemented on Kitware is pleased to present this year's Insight Journal the GPU with GLSL shaders in order to achieve interactive special edition of the Source which features several of the rendering. Solely projected depth information is required strongest Insight Journal submissions from 2010. The Insight to compute the shading function, which is then applied to Journal was designed to provide a realistic support system the colored scene image. EDL can therefore be applied to for disseminating scientific research in the medical image any kind of data (isosurfaces, streamlines, point sprites, etc.), processing domain. Recognizing the need for a mechanism except those requiring transparency-based rendering. The whereby the medical image analysis community can collec- implementation of this technique opens the way to new tively share their ideas, code, data and results, Dr. Luis Ibàñez efficient post-processing effects in Paraview. has worked with the ISC and championed the open-science cause to make the Insight Journal a reality. By providing a platform for open-access scientific publica- tion, Kitware continues to foster its commitment to the open-science community. To continue celebrating this cause we annually publish this special Insight Journal Edition of the Source. Anyone may submit their work to the Insight Journal by registering, for free, at insight-journal.org; perhaps your work will be featured here next year! The Kitware Source is just one of a suite of products and services that Kitware offers to assist developers in getting the most out of its open-source products. Each project’s website contains links to free resources including mailing lists, documentation, FAQs and Wikis. In addition, Kitware supports its open-source projects with technical books, user’s guides, consulting services, support contracts and training Figure 1: An image produced on EDF physical simulation data courses. For more information on Kitware’s suite of products rendered with point sprites illustrates the effect of EDL (left) and services, please visit our website at www.kitware.com. compared to basic Phong shading (right). There should also be a marked performance improvement During this review of the toolkit, more explicit types were for users dealing with large multi-block datasets. We have introduced to represent the concepts of: cleaned up the rendering pipeline to better handle com- • IndexValueType: The component of the index that posite datasets, avoiding the appending of all blocks into a addresses a pixel in an image. single dataset as was done previously. • OffsetValueType: The difference between two such To better utilize multiple cores on modern mutli-core indexes. machines, by default ParaView can now run using a parallel • SizeValueType: The type used for counting number of server, even for the built-in mode. This enables the use of pixels or number of elements in an array. all the cores for parallel data processing, without requiring the user to start a parallel server. ParaView binaries will also With this modification, ITKv4 reaches another milestone to be distributed using an MPI implementation, making this provide support for the very large images that are becoming feature available to users by simply downloading the bina- the standard in fields such as microscopy, computer vision ries. Since this is an experimental feature, it is off by default, and remote sensing. but users can turn it on by checking the Auto-MPI checkbox in the application settings dialog. VTK 5.8 VTK 5.8 has had a major revamp of its wrapping infrastruc- For developers interested in adding support for advanced ture. The Python wrappers have been expanded to expose multi-pass rendering algorithms to ParaView, this release more of the C++ interface. Several special VTK types like includes a major refactoring of ParaView's rendering pipe- vtkVariant have been wrapped in Python, and the constants line. View and representations have been redesigned and defined in the header files are automatically wrapped. users should see improved performance in client-server Python method calls now support unicode strings, two- mode from reduced interprocess communication during dimensional arrays, pass-by-reference, default argument rendering. values, and typedef argument types. Additionally, the 3.10 release includes several usability enhancements. 3D View now supports smart context menus, accessed by right-clicking on any object in the 3D View to change its color, representation, color map and visibility. Left-clicking on an object in the 3D View makes it active in the pipeline browser. Within the spreadsheet view, sorting is now supported and an advanced parallel sorting algo- rithm ensures that none of the benefits of the spreadsheet view, such as streaming and selection, are sacrificed. Python tracing and macro controls are now exposed from the main menus, no longer hidden under the Python shell. For an exhaustive list of the new features and bug-fixes, please refer to the change log at: http://www.paraview.org/Bug/ changelog_page.php. CMAKE 2.8.4 The CMake version 2.8.4 release is scheduled for January 2011. The roadmap for bug fixes proposed to be included in the 2.8.4 release is at http://public.kitware.com/Bug/ roadmap_page.php. The change log for bugs that have already been fixed is at http://public.kitware.com/Bug/chan- gelog_page.php. Please try out the release candidates for CMake 2.8.4 to build your projects as they become available in January. Let us know on the CMake mailing list if you run into anything 2D histogram of the value of sin(x)*cos(y) with HSV color mapping unexpected. In addition, the code underlying the VTK wrapper-generators ITKV4 ALPHA-04 RELEASE has been revamped. It now includes a more-or-less complete In this alpha release of ITKv4, a large-scale revision of the C++ parser that no longer requires unwrappable sections of toolkit was made to replace the use of "long" and "unsigned the header files to be marked with BTX/ETX. The internal long" types with integer types suitable for addressing ele- data structures used by the wrapper generators have been ments of memory. expanded and cleaned up, so future enhancements of the wrappers will be much easier to achieve. The previous use of explicit "long" types in ITK was making it impossible to use images larger than 4GB in Windows The charts have been updated to make use of the 2D scene, 64bits platforms, since the "long" types in Windows 64bits with new text rendering and improved OpenGL render- only have a 32bits capacity, and therefore can only address ing routines. New chart types such as pie charts and 2D up to 4GB of memory. This problem does not exist in Linux histograms have been added. The scatter plot and parallel and Mac 64 bits platforms where "long" types are 64bits in coordinate plots can be colored by a scalar array, and mul- size. However, replacing "long" with "size_t" and its associ- tiple charts can now be rendered in the same scene. The old ated types prepares the toolkit for the larger number of bits Qt based charts have now been deprecated and are not built processors that we anticipate to have in the next decade. by default.
2 MEDICAL IMAGE ANALYSIS WITH ITK ON APPLE IOS
Mobile computing devices are becoming increasingly impor- tant and prevalent. Apple’s mobile devices, the iPod touch, iPhone, and iPad, are notable examples. At the same time, ITK is becoming increasingly popular; implementing a large number of basic and advanced image processing, segmen- tation, and registration algorithms; and benefiting from a tremendous growth through open-source code contributed by leading research groups and individuals from around the Parallel coordinates plot world. The time is right to seriously explore the marriage of these two exciting parallel developments: ITK for medical VTK now also provides modern CMake Exports. Inclusion image analysis and iOS for mobile computing devices. of the VTKTargets.cmake file allows easy access to all VTK targets within your own projects that build against VTK. The ITK ON IOS UseVTK.cmake file has also been updated to automatically In the 2010 July-December issue of the Insight Journal [1] include VTKTargets.cmake. we were happy to share with the scientific community the detailed steps needed for building ITK on iOS and creating a basic ITK app on the iOS, which demonstrated successful basic integration. However, our long-term goal has been to open up mobile computing devices to the whole of ITK. We wanted to make it much simpler for an iOS app developer to take advantage of the full ITK functionality. Since images are at the core of such development, the first obstacle we needed to overcome was the lack of a well-defined interface in ITK for the iOS for handling images. Our next mission was clear: we needed to create an ITK Image IO Interface with Apple iOS. On September 13th, 2010, we were excited to release exactly that to the community [2]. ITK IMAGE IO FOR IOS While in other operating systems the images are specified by a path to the image file itself (e.g. @”/lib/itk/sampleimages/ mri.jpg”), the iOS software development kit (SDK) uses the UIImage object which contains all the image pixel data and metadata. In iOS, the file paths themselves are abstracted even from the programmer and only the file data can be manipulated. Both image reading and writing is done through Apple’s built-in interface. The interface either loads an image from Stacked bar chart displaying energy consumption the photo album library and returns its data as a UIImage, or saves the data in a UIImage object back in a file in the album CDASH 1.8 library. Hence, any interface that deals with image IO must CDash, the open-source, web-based software testing server be able to probe and manipulate UIImage instances. Based has had another major release since February 2010. CDash on the UIImage class, we have developed itkiOSImageIO, aggregates, analyzes and displays the results of software the necessary ITK class that provides the interface with the testing processes submitted from clients around the world, repository of images stored on iOS devices. conveying the state of a software system to allow continu- ous improvements in its quality. This new release fixes more ITK Image IO Methods and Interface than 20 bugs and adds new features including: The main behaviors our class had to implement were: • Support for asynchronous submissions • Reading image metadata (dimension, color scheme, • Support for CDash@Home: central management of builds tropicity, etc.) • Automatic removal of builds based on group type • Reading image pixel data • Better code stability and improved coverage • Writing image metadata • Better submission checking with CTest • Writing image pixel data • Better report of submission errors As a result, we implemented several critical methods in Additional information about CDash 1.8 is available at itkiOSImageIO: www.cdash.org. The virtual void ReadImageInformation() is a method that Need a quick way to get started with CDash? We can host reads the properties (dimensions, color scheme, tropicity, your project for free at http://my.cdash.org. etc.) of the current image in question.
3 The virtual void Read(void* buffer) is a method that reads Second, create the context with the command: the current image data (the actual value at each pixel). CGContextRef theContext = CGBitmapContextCreate The virtual void WriteImageInformation() is a method that (buffer, width, height, bitsPerComponent, writes the image information to the current image. Note this bytesPerRow, colorSpace, theBitmapInfo) does not write the pixel data yet. Notice that the first argument, buffer, is the data buffer that The virtual void Write(const void* buffer) is a method that needs to point to the data. writes the image data into the image file. This writes the Third, draw the image of interest into the context set up actual pixel data. using the command: Figure 1 demonstrates the data flow offered by the image IO class: CGContextDrawImage(theContext, CGRectMake(0, 0, width, height), theCGImageRef) This draws the image specified by theCGImageRef into the- Context, with the data buffer pointing to the data of the newly drawn image. Fourth, release the context, since we are only interested in the data pointed to by the variable 'buffer'. IMPLEMENTING WRITING METHODS To create an image file in ITK, the metadata is written as the first few blocks of memory using the WriteImageInformation method, followed by writing the image pixel data using the Write method. However, on the iOS, the UIImage class should be used to write the image, which in turn requires access to not only the image pixel data, but also the meta- data. Therefore, the Write(const void* buffer) method Figure 1: The role of itkiOSImageIO as the mediator between is responsible for organizing all the pertinent image data Apple’s iOS and ITK. (pixel data and metadata) and saving it into the appropriate place on the iOS. The following steps highlight the image IMPLEMENTING READING METHODS writing procedure on the iOS. ReadImageInformation() is the first nontrivial method to First, all image metadata is collected. Second, the image is implement. The necessary image properties that need to created using the command: be read via this method are the dimensions of the image (width and height), the number of bits used to store each CGImageRef theImageRef = CGImageCreate(width, height, color component value, as well as the interpretation of these bitsPerComponent, bits, e.g. 8 bits per pixel for a 256-level grayscale image or 32 bitsPerPixel, bits per pixel for Red, Green, Blue, and Alpha transparency bytesPerRow, (RGBA) values of color images. colorSpace, bitmapInfo, Fortunately, these necessary components are accessible theDataProvider,nil, via Objective-C queries to the CGImage object, which can shouldInterpolate, theIntent) be obtained from the UIImage object via the method call [theUIImagePointer CGImage]. This call returns a reference This creates an image with all the required properties. The to a CGImage object which contains all the meta data of last three arguments in this command are not critical, but the image. This metadata can be queried directly through further information can be found at Apple’s Developer methods implemented in the CGImage class, resulting in website [4]. most calls such as image dimensions and number of bits per Third, a new UIImage object is created using the CGImage pixel being trivial. from the previous step using the command: The method virtual void Read(void* buffer) needs the data UIImage* outputImage = buffer in the argument to be filled with the pixel data. [UIImage imageWithCGImage:(theImageRef)] Unfortunately, the image data is not easily accessible to the programmer. There is no known way to directly extract the Fourth, the image is saved into the iOS image library using image data from the UIImage object or from the CGImage the command: object. Instead, we used a code snippet posted publicly UIImageWriteToSavedPhotosAlbum(outputImage, online [3] . In essence, this code creates a new image context NULL, NULL, NULL) in Objective-C, then redraws the image in the new context with the data buffer as a pointer to the context data. The Again, the last three arguments are not critical, but more following steps expose the details. information about this method can be found on Apple's Developer website. First, obtain the metadata information required to create the context. This includes the number of bits per component, EXAMPLE “HELLO IOS IMAGES” the number of bits per row, the color space information, Here, we show how we used the new itkiOSImageIO class endianness, the existence of an alpha (transparency) channel to read an image and make use of existing ITK classes to or not, and the width and height in pixels of the image. filter the image, followed by using the itkiOSImageIO class
4 to write the resulting image to the iOS image library. For determine at run-time the file format (typically based on brevity purposes, we will only show one set of experiments the file extensions) and invoke the proper image IO code which uses an RGB image, but further results can be found accordingly. Implementing this mechanism for the iOS, given in the online publication [2]. the iOS constraints on obtaining image information, is left for future work. In order to demonstrate the effectiveness of the itkiOSIm- ageIO class when dealing with color images, we have chosen For medical images in particular, the physical dimensions of the color image in Figure 2(a) from the public domain Visible each pixel are quite important. Unfortunately, when reading Human Project provided by the US National Library of an image, iOS assumes only isotropic pixels without any Medicine. We converted it into a grayscale image using ITK’s physical dimensions assigned to it. Nevertheless, the itk::RGBToLuminanceImageFilter, and performed binary programmer may set the pixel size of the image using filtering on it using itk::BinaryThresholdImageFilter. When itkiOSImageIOInstance->SetSpacing(0,xSpacing) for the reading and writing the image, the code ran error-free and horizontal pixel spacing and itkiOSImageIOInstance- the results can be seen in Figure 2(b) and 2(c). >SetSpacing(1,ySpacing) for the vertical spacing. Further exploration of working with physical units of pixel resolu-
tions, image offsets, direction cosines, etc., on the iOS remains an important future goal. Although very popular, the iOS is not the only mobile OS. We hope our work will spark parallel efforts that will bring ITK to other mobile platforms such as Google’s Android. With mobile operating systems running ITK, it would be interest- ing to profile the processing speed and memory usage of ITK on these devices and explore the possibility of using several of those as thin clients for server based image analysis and processing. Finally, it is important to have serious discussions with doctors to better appreciate the clinical applications that would benefit most from medical image analysis on mobile devices and work together on developing apps targeting specific tasks. Figure 2: Reading and writing a color image using itkiOSImageIO. (a) the original image, (b) its grayscale converted copy and REFERENCES (c) its filtered copy. [1] Boris Shabash, Ghassan Hamarneh, Zhi Feng Huang, and Luis Ibanez. ITK on the iOS. Insight Journal, July-December:1-9, 2010 FUTURE FOR MOBILE MEDICAL IMAGE ANALYSIS [2] Boris Shabash, Ghassan Hamarneh, Zhi Feng Huang, and Luis This work has been motivated by the importance of medical Ibanez. ITK Image IO Interface with Apple iOS. Insight image analysis in health applications, which is increasingly Journal, July-December:1-16, 2010 being performed using the ITK library, and by the ubiquitous [3] http://stackoverflow.com/questions/448125/ mobile devices, in particular Apple’s iOS devices, with their [4] http://developer.apple.com increasingly faster processing, larger storage, and exquisite multi-touch displays. We developed itkiOSImageIO, the Zhi Feng Huang received his B.S in Computing necessary ITK class to interface with images on iOS devices. Science from Simon Fraser University in However, many important steps still need to be taken to 2009. He is a Master’s student in the SFU make mobile medical image analysis a popular reality. Vision and Media Laboratory performing One of the important extensions to our work would be to research on computer vision and machine facilitate reading, writing, and processing of 3D or higher learning and is funded by the NSERC of dimensional medical images using ITK on iOS devices. There Canada through the CGS Master scholarship. exist several iOS apps that work with and visualize 3D Boris Shabash is a Master's student at the medical images. These 3D images are handled by the app School of Computing Science at Simon Fraser itself and not via the iOS photo library. The most suitable University and a Bachelor of Health Sciences approaches for working with high dimensional images using graduate from the University of Calgary. His ITK on the iOS remain to be seen. current research focus is RNA structure visu- The second foreseeable development is supporting C++ alization and human computer interaction libraries other than ITK, mainly the Visualization ToolKit in the field of visual bio-informatics. (VTK), ITK’s visualization homologue. In the application Ghassan Hamarneh is an Associate Professor presented here, we used the UIImageView object to display at the School of Computing Science, Simon the images before and after processing. VTK, on the other Fraser University, Canada and is the co- hand, offers much greater image visualization capabilities director and founder of their Medical Image and works especially well with 3D images and spatial objects Analysis Lab. He obtained his doctoral and produced by ITK. master’s degrees from Chalmers University Another direction for further development is related to ITK’s of Technology in Sweden, and his bachelor’s pluggable object factories. This factory mechanism allows degree from the University of Jordan. His primary research the ITK ImageFileReader and ImageFileWriter functions to interest is in medical image analysis.
5 ULTRASOUND AND ITKV4
The most common imaging mode in diagnostic medical ultra- sound imaging is the B-Mode (Brightness-Mode). Recently, a manuscript [1] along with corresponding source code was submitted to the Insight Journal that describes classes for the Insight Toolkit (ITK) [2] to create B-Mode images. This article summarizes the Insight Journal paper and also discusses challenges and successes that were encountered in the development of the software. We will examine how the ongoing activities for ITKv4 address some of these issues. Calculation of the brightness from a raw received ultrasound signal amounts to approximating the envelope of the signal. The envelope is often calculated with the analytic signal,
f A (x) f (x) i f H (x) where f H (x) is the Hilbert Transform− of f (x). Given the analytic signal, the envelope is simply
A (x) √ f 2 (x) + f 2 (x)
The analytic signal is often calculated with the Fourier Transform, Figure 2: Gummy bear slices taken with the VisualSonics RMV710B 25MHz transducer. a) axial-lateral, scan converted slice, b) axial- F F . [1 + sgn elevational slice where lines in the elevation direction are created A( ) ( ) ( )] by a stepper motor, c) lateral-elevational, "C-Plane", slice, and d) tri-plane view. Images rendered with QGoImageCompare [3]. A number of classesξ were implementedξ to performξ the Fast Fourier Transform (FFT) along a single direction of an image, the direction of the ultrasound beam's propagation. We also presented code to perform scan conversion. This includes classes to resample data from a polar coordinate system, which is commonly encountered with curvilinear, phased array, or rotated single-element transducers. Example high frequency ultrasound B-Mode images of the object shown in Figure 1 are rendered in Figures 2 and 3. The scan conversion and streaming capabilities described in the article were required to create this image.
Figure 3: Volume rendering of the gummy bear B-Mode. Image Figure 1: Gummy bear photo. created with Paraview [4].
6 GPGPU'S AND THE FFT In its current state, this filter does not allow streaming. We Since the FFT is the most computationally expensive part introduced a modification to this filter that would allow for of the algorithm, we tested three different FFT implemen- streaming as long as the transform associated with it is linear. tations: VNL, FFTW, and an OpenCL implementation. We This feature may be helpful to ITK in general, and it could be found that the FFTW was faster than VNL, but the OpenCL incorporated into ITKv4. However, to retain its behavior in was slower in spite of the hype surrounding general purpose this example, the itk::Transform classes should change their GPU (GPGPU) computing for scientific analysis. IsLinear() method to return an array of booleans per dimen- sion as opposed to a single boolean. This is required because The speed deficits were likely related to our implementation: the scan conversion transform is non-linear the r- plane. the image buffer needed to be transferred to and from the GPU when performing the forward and inverse transform. VERSION CONTROL θ For GPGPU computing to be successful, additional infrastruc- The submitted code of the repository was also a Git reposi- ture is required in ITK to prevent this memory transfer. tory with its "origin" pointing to Gitorious [8]. This allows reviewers to easily fix or extend the work. We have been There is a proposal to add GPGPU capabilities to ITK in impressed by the power of Git and are happy to see it being ITKv4 [5]. This proposal addresses the problem of keeping adopted as the version control system for ITKv4. the image buffer on the GPU as long as possible along with abstracting out some of the difficulties experienced when The article was our first experimentation with Git submod- writing GPGPU code. Writing OpenCL code requires extra ules. This proved to be an effective method to manage logical build steps and additional memory management, and it sections of code, sometimes from disparate sources. It seems precludes the native use of C++ templates. When develop- that ITKv4 will also soon take advantage of Git submodule ing our code, we used the C++ OpenCL class wrappers that capabilities with the modularization project [9]. There are have since been incorporated into the OpenCL 1.1 standard. rumors that these capabilities will be used to improve the However, these classes do not provide a very object-oriented accessibility of work published in the Insight Journal. This interface; they just provide destructors that ease memory increased organization of such a complex project will allow management. In our opinion, writing OpenCL code, a C users to find what they are looking for and contributers to language with extensions, is considerably more work than make improvements with less effort. vanilla C++. FUTURE WORK Implementing GPGPU support in ITK will take substantial We hope to release additional tools for basic ultrasound work, but supporting this class of hardware means ITK can research with ITK including the release of a block-matching stay relevant in the parallel future. framework for deformable image registration. FILE FORMATS ACKNOWLEDGEMENTS Some manufacturers of clinical ultrasound systems provide The authors would like to thank the NIH for their funding methods to access the radiofrequency (RF) data collected on with grants T90DK070079 and R90DK071515. We would also their systems. A considerable problem is that each system like to thank the many developers of the Insight Toolkit. uses its own file format, and the format is proprietary. An astute examiner of the CMake configuration included with REFERENCES the project will notice that there is reference to optional [1] McCormick, M. "An Open Source, Fast Ultrasound B-Mode support of a few of these formats. Software Implementation for Commodity Hardware." Insight Journal. May 2010. http://hdl.handle.net/10380/1338. Unfortunately, we could not include the code for the ImageIO [2] Insight Segmentation and Registration Toolkit. itk.org/. class and supporting libraries that were developed. While it [3] Perrot-Audet A., Mosaliganti K., Gelas A., Rannou N., took considerable work to create readers for the complex Souhait L., Megason S. "A simple Qt based comparison formats and we would not like to see other researchers program for ITK and VTK images." Insight Journal. August repeat this process, we lack the resources [6] to release this 2010. http://hdl.handle.net/10380/3196 code. In order to create the readers, the companies provided [4] ParaView is an open-source, multi-platform data analysis documentation they considered proprietary. and visualization application. http://paraview.org/. [5] Won-Ki. GPU Acceleration. Fall ITKv4 2010 Meeting. To distribute our data, we converted the images to some of www.itk.org/Wiki/images/3/3a/GPU-ITKv4.pdf the standard supported ITK formats. We found it difficult to [6] Zevon, W. "Lawyers, Guns, and Money." Excitable Boy. find a format that would support our additional metadata. 1978. We tried the MINC format, but we did not have success and [7] HDF5 ImageIO. www.itk.org/Wiki/Proposals:HDF5_ImageIO ended up using the NRRD format. The NRRD format was not [8] Gitorious provides free open source infrastructure for open ideal, however, because it only supports string fields. source projects that use Git. There is a proposal to add HDF5 support [7], which is a flex- http://gitorious.org/ultrasound-b-mode/b-mode ible format that has gained considerable traction in general [9] Hoffman, B. Liu, Xiaoxiao. "Modularization." Fall ITKv4 scientific computing applications. Support for adding HDF5 2010 Meeting. http://www.itk.org/Wiki/images/f/ff/ITK- fields to an image's MetaDataDictionary would be beneficial Iowa-Nov-2010-Modularization.pptx for situations like this. Matthew McCormick is a PhD. candidate STREAMING at the University of Wisconsin-Madison Large 3D ultrasound images require data streaming to prevent working on his thesis in medical ultrasound the filling of system memory. In general, ITK's excellent characterization of carotid plaques. streaming infrastructure makes this an easy task. However, scan conversion requires the use of itk::ResampleImageFilter.
7 provides the necessary conversion allowing the user to utilize NEW FUNCTIONALITIES FOR the itk::Statistics::SampleToHistogramFilter [6] for histogram generation of mesh scalar values. Here is the example on SPHERICAL DEMONS REGISTRATION how to use this filter to generate a ListSample from the input mesh. ITK has recently added the ability to register surfaces in the spherical domain based on the quad edge mesh data typedef itk::MeshToListAdaptor
MESH TO LIST ADAPTOR Figure 2 demonstrates the effectiveness of the histogram The itk::MeshToListAdaptor filter can generate a list of matching mesh filter on two simple meshes as shown in vectors from the scalars assigned to the input mesh [7]. It Figure 2(a) and Figure 2(b), along with their scalar values in
8 the range of [0.0, 1.0] and [0.5, 1.0] respectively. The output the cell is added to the calculation of A or B. However, if only mesh with histogram matched scalars is shown in Figure 3(c). one point of that cell is associated with the label, a third of the triangle area is added. If there are two points associated with the label, then two thirds of the area is added.
Figure 2: The effect of histogram matching mesh filter. The scalar values of meshes in this figure are colored by the sidebar shown on the right. (a) the input/source mesh of the filter. (b) the reference mesh of the filter. (c) the output of the filter. In order to calculate the Dice or Jaccard index, the value of WARPING MESHES the label that needs to be evaluated at both of the input The itk::WarpMeshFilter warps a mesh using an input meshes should be specified by the user. deformation field [9]. It was originally designed to warp anatomical labels that are mapped from the atlas space typedef itk::QuadEdgeMeshSimilarityCalculator onto a subject surface. The WarpMeshFilter takes three < InputMeshType1, inputs: 1) input mesh, 2) reference mesh and 3) deforma- InputMeshType2 > SimilarityCalculatorType; SimilarityCalculatorType::Pointer similarityCalculator tion field. The input mesh contains the information of initial = SimilarityCalculatorType::New(); point positions while the reference mesh contains the scalar similarityCalculator->SetLabelValue(1); information that is going to be interpolated. The points on similarityCalculator->Update(); the reference mesh could be the same as the input mesh or This filter can be used to evaluate the overlapping of warped not. The deformation field can be provided as a mesh with labels on the subject’s cortical surface with the manual labels vectors (instead of scalars) associated with its points. The delineated by raters. input mesh and deformation field should have the point-to- point correspondence with each other. CONCLUSIONS AND FUTURE WORK Each point on the input mesh is moved according to the The presented filters were originally created for the purpose of developing an automated cortical labeling algorithm corresponding vector on the deformation field (pnew=pin+d). A new scalar value is calculated by using interpolation on the using ITK. However, many of these filters could be used in reference mesh and mapped back onto the input mesh. An several applications. These filters are being added to a new major update to ITK (version 4). This will provide users with interpolator is needed by the filter since pnew is likely to fall inside of a cell and not directly on a vertex. Figure 3 shows the necessary infrastructure to develop novel, based surface the results of applying the warp mesh filter. registration algorithms using ITK. ACKNOWLEDGEMENTS This work is funded by NIH/NINDS grant NS050568 and thanks to Luis Ibáñez for his help and discussion. REFERENCE [1] L. Ibanez, M. Audette, B.T.T. Yeo, and P. Golland. Rotational Registration of Spherical Surfaces Represented as QuadEdge Meshes. Insight Journal. 2009. [2] L. Ibanez, M. Audette, B.T.T. Yeo, and P. Golland. Spherical Demons Registration of Spherical Surfaces. Insight Journal. 2009. Figure 3: Shows (a) the input fixed mesh and the reference moving [3] B. T. T. Yeo, M. Sabuncu, T. Vercauteren, N. Ayache, B. Fisch, mesh. (b) the deformation field on the mesh. Vectors are shown as and P. Golland. Spherical demons: Fast diffeomorphic arrows on the mesh with the magnitudes on the associated color landmark-free surface registration. IEEE TMI, 29(3): 650 – map. (c) the output of the filter. The scalar values on (a) and (c) are 668, 2009. between 0.0 and 1.0 and colored from blue to red. [4] W. Li and V.A. Magnotta. Assign Scalars Mesh Filter. Insight Journal. 2010. MESH SIMILARITY CALCULATOR [5] W. Li and V.A. Magnotta. Rescale Scalars Mesh Filter. Insight The itk::MeshSimilarityCalculator filter is used to calculate Journal. 2010. the similarity of two labeled meshes [10]. The input meshes [6] L. Ibanez, W. Schroeder, L. Ng, and J. Cates. The ITK Software are assumed to have labels assigned to the surface as scalar Guide. Kitware, Inc. ISBN 1-930934-15-7, integral values and the surfaces have the same geometry. http://www.itk.org/ItkSoftwareGuide.pdf, second edition, The filter supports both the Dice and Jaccard indexes. 2005. [7] W. Li, V.A. Magnotta and L. Ibanez. Mesh To List Adaptor. The actual overlap of A and B here are calculated based on Insight Journal. 2010. the surface area of each label. If all of the points of a cell are [8] W. Li and V.A. Magnotta. Histogram Matching Mesh Filter. associated with the same label, the whole triangular area of Insight Journal. 2010.
9 [9] W. Li and V.A. Magnotta. Warp Mesh Filter. Insight Journal. ized k-dimensional Gaussian function whose major axes are 2010. aligned with the Cartesian axes, F(x;u) has the form [10] W. Li and V.A. Magnotta. Mesh Similarity Calculator. Insight Journal. 2010. F(x;u k/2 | |1/2)) exp[-(1/2) (x T 1(x
Wen Li is currently pursuing her PhD in the ) = (s / ((2π) ∑ - μ) ∑- - μ)] department of Biomedical Engineering at where s is a scaling factor, µ is the mean (center) vector, and the University of Iowa and is employed as a is a diagonal matrix whose i-th diagonal element is the standard deviation i of the Gaussian function along the i-th graduate research assistant in the depart- ∑ ment of Radiology. She got her B.S. and M.S. dimension. For this function, the parameter vector u is equal degrees at Shanghai Jiao Tong University in to {s, i, ..., k, µ1,…, µk}.σ In ITK, the class itk::GaussianImageSource China and spent three years as a lecturer and generates images whose intensities are defined by this func- research assistant at the Science and Technology University tion whenσ σ its Normalized flag is set to 1. of Shanghai. Her current research is automated surface par- Certain ITK image filters may also be considered parametric cellation of the human cerebral cortex using MR images. models of image pixel values. For these filters, the function is Vincent Magnotta is an Associate Professor of the form F(x;u,I(x)) where u is a parameter vector as before of Radiology, Biomedical Engineering, and and I(x) is the image input for the filter. In some applications Psychiatry at the University of Iowa. He is it may be desirable to optimize the filter parametersu . For currently a developer on the ITKv4 refactor- example, the itk::MultiplyByConstantImageFilter might be ing. He is also working on two open-source used to find an optimal intensity scaling factor for an image projects that build upon ITK and VTK: with respect to a particular objective function. BRAINS (http://www.nitrc.org/projects/ While ITK has parametric image sources, no unified interface brains) and IA-FEMesh (http://www.ccad.uiowa.edu/mimx/ for setting their parameters exists. This kind of interface, IA-FEMesh/). analogous to the interface currently found in subclasses of itk::Transform, would enable application of the opti- mization routines in ITK to fitting image intensity models. ADAPTING ITK FRAMEWORK TO FIT We propose a new subclass of itk::ImageSource, called itk::ParametricImageSource, to provide a suitable interface. PARAMETRIC IMAGE MODELS This class defines three methods:
Fitting models of image pixel values to images is a task virtual void SetParameters(const ParametersType& p) {}; central to certain image analysis applications. For example, in fluorescence microscopy, light from an in-focus point virtual ParametersType GetParameters() const source diffracts to an intensity pattern that approximately { matches a 2D Gaussian function. Using this observation, return ParametersType(0); methods have been developed for determining the positions } of individual fluorescent molecules by fitting a 2D Gaussian virtual unsigned int GetNumberOfParameters() const function to the image [1][2][3]. The mean (center) of the { fitted Gaussian is taken to be the location of the molecule. return 0; Localization accuracy of as low as 1.5 nanometers has been } reported using this method [3]. where the ParametersType is defined by Despite having mature implementations of the necessary algorithmic foundations, the Insight Toolkit (ITK) does not typedef double ParametersValueType; currently have a mechanism to enable the fitting of image typedef Array
10 Briefly, an optimizer is a search routine that iteratively The itk::ImageToParametricImageSourceMetric adapts a evaluates a cost function at different values of the transform delegate itk::ImageToImageMetric for the purposes of parameters until the cost function is minimized (or possibly computing the image match value. Parameters passed to maximized). A cost function assigns a value to two images, the GetValue() method from the optimizer are forwarded a fixed image and a moving image, according to how well to the itk::ParametricImageSource, which is updated before they match. The fixed image remains unmodified during the delegate computes the cost function value. Figure 1 registration while the moving image is warped according to shows how the itk::ImageToParametricImageSourceMetric the spatial transform. If the optimizer converges to a local integrates into the registration framework. minimum (or maximum), the transform and parameters We assume that the parametric image source has parameters found by the optimizer define a locally optimal warping that can accommodate any necessary spatial transformations. from the moving image to the fixed image. Therefore, the itk::ImageToParametricImageSourceMetric We would like to be able to reuse the existing subclasses of forces the delegate metric to use an itk::IdentityTransform. itk::ImageToImageMetric as cost functions in ITK to fit para- We do not make any assumptions about the size, spacing, metric image models to images. These cost functions have or origin of the fixed or moving images, so an interpola- the methods tor for the fixed image must be set using the method SetInterpolator(). MeasureType GetValue(const ParametersType &p) const; The interface for the itk::ParametricImageSource does not void GetDerivative(const ParametersType &p, currently provide a mechanism to retrieve partial derivative DerivativeType &d) const; images that can be used for computing the gradient of the itk::ImageToImageMetric in the method GetDerivative(). void GetValueAndDerivative(const ParametersType &p, Instead, we estimate the derivative via forward differences. MeasureType &v, DerivativeType &d) const; Because it may be desirable to optimize over a subset of that are called by optimizers. These methods pass their the parameters in an itk::ParametricImageSource, the itk parameters argument to an itk::Transform object assigned ::ImageToParametricImageSourceMetric class also holds to the image-to-image metric. This behavior makes it impos- a mask vector that indicates which parameters should be sible to set parameters of an itk::ParametricImageSource optimized. We call these enabled parameters. The method through the existing registration framework. To over- itk::GetNumberOfParameters() reports the number of come this limitation, we have defined a new subclass of enabled parameters. This method tells the itk::Optimizer the itk::ImageToImageMetric, called itk::ImageToParametricIma dimensionality of the parameter space that it will explore, geSourceMetric, that lets us control where the parameters so the itk::Optimizer will pass arrays containing values for are passed. only the enabled parameters to the GetValue() method of itk::ImageToParametricImageSourceMetric. However, the Much like the existing itk::ImageToImageMetric subclasses, SetParameters() method in itk::ParametricImageSource the itk::ImageToParametricImageSourceMetric is param- expects an array containing values for all the parameters. The eterized over fixed and moving image types. The moving simple solution is to retrieve the current parameters from the image type, however, is expected to be a subclass of itk::ParametricImageSource, set the active parameters passed itk::ParametricImageSource, so it is actually an image source in by the itk::Optimizer, and then pass this modified param- rather than static image data. The meaning of “moving” in eter array back to the itk::ParametricImageSource using that this context means that the image intensities are changing class's SetParameters() method. The class itk::ImageToParam values, rather than that the moving image is undergoing a etricImageSourceMetric has its own SetParameters() method spatial deformation. that takes care of this step. WRITING ADAPTER CLASSES FOR EXISTING IMAGE SOURCES A pragmatic way to implement image model fitting is to write classes that adapt existing image source to the itk::ParametricImageSource interface. As an example, the implementations of the key methods in the adapter class for the itk::GaussianImageSource are given below.
template
11 typedef itk template
First, we include the necessary software components from amoebaOptimizer-> the registration framework: SetInitialPosition(startingParameters); amoebaOptimizer->StartOptimization(); #include
12 ACKNOWLEDGEMENTS • QWidget inheritance for integration to a Qt GUI based This work was funded by NIH grant P41-EB002025-25A1. application • A manager class that takes care of visualizations proper- Cory Quammen is a software engineer and ties (creation, deletion, synchronization of several views, graduate student in the Department of look-up tables…) Computer Science at the University of North • ITK visual comparison debugging Carolina at Chapel Hill. His work involves analysis of fluorescence microscopy images. VISUALIZATION LIBRARY The library is composed by six classes, which are all Qt objects. The abstract class for QGoSynchronizedView2D and 3D Russell Taylor is a Research Professor of is QGoSynchronizedView. It is used to display a QWidget Computer Science, Physics & Astronomy, and containing a two- or three- dimensional vtkImageData* or Applied & Materials Sciences at the University itk::Image*. It provides the interface to synchronize cameras of North Carolina at Chapel Hill. He is the among several of the abstract classes' objects. co-director of the UNC NIH National Research Resource for Computer Integrated Systems QGoSynchronizedView2D(3D)Callbacks takes a list of the for Microscopy and Manipulation. His abstract 2D and 3D classes and synchronizes their cameras research interests include Scientific Visualization, Distributed by setting up callbacks. It is highly recommended to let the Virtual Worlds, Haptic Display, and Interactive 3D Computer QGoSynchronizedViewManager handle this process. Graphics. QGoSynchronizedViewManager is the high level class for the QGo 2D and 3D synchronized views and callbacks. This class deals with QGoSynchronizedViews for correct synchroniza- A LIGHTWEIGHT IMAGE tion and provides a simple interface to create, delete and COMPARISON LIBRARY synchronize QGoSynchronizedViews. For more details, we invite you to look at the Doxygen docu- mentation and the source code. QGoImageCompare was developed in the Megason Lab at Harvard Medical School and at the University of Wisconsin- LIBRARY USAGE Madison. It is a library aimed at simple comparison of For further clarification, there is a description of the basic images. The library provides an easy means to add 2D and use of the visualization library and an example to illustrate 3D image visualization and examination features to a Qt4 its functionality. application, or the bundled visualization executables can be used on their own. Code Snippets We introduce here the high-level functions for creating This project merged with Matt McCormick’s visual debugger QWidgets views and synchronizing visualizations, based on for ITK in order to provide a complete package for visualiz- the QGoSynchronizedViewManager. ing images while debugging an ITK pipeline. The developers are now from the many itk::ImageFileWriter's in their code. To develop a visualization manager object we simply create QGoImageCompare can be used in the process of developing a new manager that will take care of creation/deletion of an ITK or VTK image processing or data visualization program visualization and callbacks for us. and developers can use it to quickly visualize and compare results of filters in different parts of the pipeline. It is capable QGoSynchronizedViewManager* ViewManager = new QGoSynchronizedViewManager(); of displaying images directly from the memory while debug- ging an ITK application. Finally, QGoImageCompare can be We can now visualize and process images as necessary. used in software applications like Slicer, SNAP, GoFigure2 or Once visualization is no longer needed, remember to delete V3D in their plug-ins infrastructure [1] to compare the input ViewManager. and the resulting output of one filter. delete ViewManager; QGoImageCompare is hosted on a github repository [2]. The repository includes code source, documentation, publica- For the visualization of a VTK image, the synchronization tion, examples (in /Examples/GUI/lib/), tests (in /Testing/GUI/), manager can create visualization windows given a valid and executables / applications (in /Main/). pointer to the image and a string encoding the name of the visualization. FEATURES QGoImageCompare includes: ViewManager->newSynchronizedView("My VTK View", VTKSmartPointerToImage); • Support for ITK scalar images (see supported format for ViewManager->Update(); ITK image support details) ViewManager->show(); • Support for VTK images For the visualization of an ITK image, the synchronization • 2D or 3D images with planar views and a tri-planar (xy, xz, manager can create visualization windows given a valid yz) 3D view pointer to the image, the template argument representing • Camera synchronization for fine comparison of results the image pixel type and a string encoding the name of the • Choice of look-up tables visualization. • Pixel examination, location, index, and value at cross-hair ViewManager->newSynchronizedView
Figure 3: Illustration of comparegui example application. We use it to compare two identical datasets, in a synchronized manner, with a custom look-up table.
VISUAL DEBUGGING Here we describe the GDB-pretty-ITK project, which utilizes QGoImageCompare for debugging image-processing pipe- lines using the GNU Debugger (GDB). When creating a new ITK pipeline or filter, it is often beneficial to examine the intermediate images that are generated. In the context of a debugging, pretty-printing is the act of displaying a complex data structure in an insightful way. For example, pretty-printing a C array pointer may involve printing the elements of the array or pretty-printing a C structure may imply printing the structure’s members. In a previous article in the Source [3], we demonstrated how custom GDB Python pretty-printers could be used to launch an image viewer, which displays an itk::Image's BufferedRegion when 'print- ing' the object in GDB. In our previous work, we launched an independent child Figure 1: The diagram shows how to use the code snippets to process such as ParaView or Mayavi for each object that visualize intermediary results in an image-processing pipeline. (The was printed. Here we describe a new Python pretty-printer example can be found in /Examples/GUI/lib/pipeitkexample.cxx) based on QGoImageCompare. This pretty-printer uti- lizes the well-designed visualization and analysis tools of QGoImageCompare. It also has an advantage over the other image viewer pretty-printers because the camera view is synchronized between all images that are printed. The first time print is called on an itk::Image pointer, the pretty-printer module, called icp, launches a child process, icpGui. A separate process is needed to separate the GDB event loop and the Qt event loop. The image's information (spacing, origin, etc.) and data buffer are passed from the Figure 2: Viewers generated by pipeitkexample, showing (a)the Python module to the GUI via Qt's inter-process communi- input image, (b) the filtered image, and (c) the thresholded image. cation (IPC) capabilities. Subsequent images are sent across
14 the same IPC channels and the view is synchronized between Lydie Souhait is a Research Associate in the images. Simultaneous synchronized zooming, for instance, Megason Lab at Harvard Medical School can be very helpful in image analysis. where she is participating in the develop- ment of GoFigure2. She is currently in charge The GDB-pretty-ITK project is still in the alpha stages, but it of the Database and GUI components in is usable and may be helpful in your adventures with ITK. GoFigure2. We have tried to lower the barrier to installation and usage, although there is admittedly a way to go. A set of installation instructions has been placed on the ITK wiki [4]. Instructions Kishore Mosaliganti is a Research Fellow in exist for the QGoImageCompare pretty-printer along with the Megason Lab at Harvard Medical School the rest of the system. Note that the entire system must be where he is currently developing algorithms installed to use the icp pretty-printer. We also developed a for the extraction of zebrafish ear lineages set of CMake modules to ease registration of the ITK pretty- using confocal microscopy images to be printers in your project [5]. included in GoFigure2. An example use of the icp pretty-printer is distributed as a CTest test within the source tree. This example includes a simple executable and a GDB script of commands that can Sean Megason is an Assistant Professor in be interactively entered into the debugger. the Department of Systems Biology at Harvard Medical School. The Megason lab REFERENCES wants to understand how the code in the [1] Chen, S. Insight Toolkit Plug-ins: Volview and V3D, Kitware genome is executed to turn an egg into an Source, October 2010. embryo. [2] QGoImageCompare: Source code of QGoImageCompare. https://github.com/gofigure2/QGoImageCompare. [3] McCormick, M. Visual Debugging of ITK. Kitware Source. April 2010. http://kitware.com/products/html/ VisualDebuggingOfITK.html [4] ITK/GDBPretty. The ITK Wiki. http://www.itk.org/Wiki/ITK/ GDBPretty. KITWARE NEWS [5] gdb-pretty-itk-cmake: CMake scripts to register GDB pretty- printers for ITK on targets. http://gitorious.org/gdb-pretty/ gdb-pretty-itk-cmake. IEEE SUPERCOMPUTING 2010 ParaView developers from Kitware and elsewhere were hard at work at the IEEE Supercomputing (SC10) conference in ACKNOWLEDGEMENTS November. Andy Bauer and Pat Marion along with Sandia's This work was supported by NHGRI P50 HG004071 and Ken Moreland and Nathan Fabian presented a tutorial on NIDCD R01 DC010791. ParaView's new coprocessing framework, through which Antonin Perrot-Audet is a Ph.D. student at ParaView can be connected to a running simulation to visu- the CREATIS Lab, University of Lyon in alize results in-situ. Dave DeMarle set up a demonstration France, and in collaboration with the of ParaView's Manta ray traced rendering plugin, which did Megason Lab at Harvard Medical School, interactive multi-million polygon count software render- working on his thesis in registration of cell ing on a four node Intel cluster at the Intel booth and also lineages. worked behind the scenes with John Patchett, Chris Mitchel and Joshua Wu from LANL setting up a demonstration of vortex core detection in ParaView on high resolution 3D Matthew McCormick is a PhD. candidate at ocean models at the LANL booth. the University of Wisconsin-Madison working on his thesis in medical ultrasound KITWARE RECEIVES HPCWIRE READERS’ AND characterization of carotid plaques. EDITORS’ CHOICE AWARDS At SC10, Kitware was presented with HPCwire's Editors' and Readers' Choice Awards for Best HPC Visualization Product or Technology for ParaView. Berk Geveci was in attendance Arnaud Gelas is a Research Associate in the to receive the prestigious awards from Tom Tabor, the pub- Megason Lab at Harvard Medical School lisher of HPCwire. where he is in charge of the GoFigure2 The HPCwire Readers’ and Editors’ Choice Awards are project. determined through online polling of the global HPCwire audience, along with a rigorous selection process involving HPCwire editors and industry luminaries. The awards are an annual feature of the publication and constitute prestigious Nicolas Rannou is a Research Associate in recognition from the HPC community. "HPCwire readers are the Megason Lab at Harvard Medical School among the most informed in the HPC community and these where he is participating in the develop- awards represent which HPC-related companies are making ment of GoFigure2. He is currently in charge the biggest impacts in mind-share within this community,” of the Visualization component in said Tom Tabor. "These awards send a strong message to the GoFigure2. recipients that those in the global HPC community recognize their work, and consider their efforts meritorious.”
15 Kitware is honored to accept these awards on behalf of the attracts about 800 attendees and consists of several con- global ParaView community that has made ParaView the ferences focused on various aspect of visualization: IEEE success it is today. Key members of this community, who Visualization, IEEE Information Visualization and IEEE have provided invaluable technical expertise and support, Visual Analytics Science and Technology (VAST). Once again include Sandia National Laboratories, Los Alamos National Kitware was a Gold Supporter for VisWeek. From Kitware Laboratory, the Army Research Laboratory, the Swiss Lisa Avila, Utkarsh Ayachit, Jeff Baumes, Berk Geveci and National Supercomputing Centre, and Électricité de France. Will Schroeder attended. One of the highlights of the conference was the full-day tutorial "DIY Visualization Applications," taught by several groups including the University of Utah, Sandia National Labs, the University of Münster, and Kitware’s Berk, Jeff and Utkarsh. The course presented a mix of visualization tools, focusing on customization for particular application needs and explained the utility of Titan for quickly building useful informatics applications. One example program used text analysis and Protovis visualizations to enable understanding, searching, and navigating large collections of documents. In just a few minutes, it processed the entire multi-gigabyte VisWeek 2010 paper dataset, demonstrating a very relevant example to the audience. Kitware held a Birds-of-a-Feather meeting, dedicated to our open source communities. In addition to presentations given by Kitware, Terry Yoo (NLM) gave an update on ITKv4, Ken Moreland (Sandia) discussed the ParaView CoProcessing Library, Jim Ahrens (LANL) described Visualization and Analysis for Massive Datasets with VTK/ParaView, and Erik Anderson (Utah) provided an update on VisTrails. Tom Tabor, the publisher of HPCwire, with Berk Geveci VTK ARCHITECTURE REVIEW BOARD MICCAI 2010 The VTK Architecture Review Board (ARB) was formed Kitware was actively involved in the 2010 Medical Image approximately one year ago. The ARB consists of a group of Computing and Computer-Assisted Intervention (MICCAI) individuals whose goal is to advance the technology in VTK conference held in Beijing, China. Kitware’s Michel Audette, by providing direction and oversight to the development of Stephen Aylward, Andinet Enquobahrie and Luis Ibáñez VTK. The ARB was formed to balance the design goals of attended the conference. the growing VTK community, ensuring that changes benefit the community as a whole. The two primary functions of Kitware sponsored the event for the first time this year and the ARB are to maintain short- and long-term roadmaps of co-sponsored the Young Scientist award with NDI, with VTK and to make decisions relative to high-impact code and Stephen Aylward present to hand out the awards. In addi- design changes. To learn more about the VTK ARB, please tion, Kitware taught and organized several tutorials and see the wiki http://www.vtk.org/Wiki/VTK/ARB, which con- workshops. tains a current list of VTK ARB members and meeting notes Stephen was an instructor at the “From MICCAI Algorithms from the past year. to Clinical Translational Tools: The NA-MIC Platform” work- FOSS EDUCATION MEETING shop, which addressed the critical step of maximizing the Kitware is actively involved in educating the next genera- long-term positive impact of MICCAI algorithms by fostering tion of engineers on the legal, technical and social concepts their deployment, validation and concrete use by the clinical of Free and Open Source Software. One of the highlights research community. of this effort is our fall semester course at the Rensselaer Luis organized the ITKv4 Mesh Hackfest, designed to make Polytechnic Institute (RPI) on "Open Source Software progress on the refactoring of ITK’s Mesh classes, in which Practices." We also explore collaborations with other higher Michel actively contributed long into the night. Luis also education institutions at the local and national level. In this taught a tutorial on Microscopy Image Analysis. The other context, on March 30th, we held a meeting with university collaborators included: Hanchuan Peng and Fuhui Long from professors from the Albany area in order to exchange ideas HHMI - Janelia Farm Campus, Sean Megason from Harvard on how we could collaborate to ensure that FOSS-related University, Jense Ritch from GE Research and Tianming Liu activities that take place in one campus can benefit students from University of Georgia. The tutorial covered state-of- on other campuses. We also exchanged ideas on how the the-art microscopy image analysis and included hands-on curriculum of computer science and software engineering exercises using the V3D application (developed at HHMI) could be improved in order to better match the needs of and a set of ITK plugins. The teaching materials are available the marketplace. Topics of great interest were: the software under Creative Commons License 3.0 at the MIDAS collec- development process, software quality, and large project tion: http://midas.kitware.com/collection/view/30. management. The meeting was attended by Prof. Seth Chaiken from SUNY Albany, Prof. Kristina Striegnitz and Prof. VISWEEK 2010 David Hemmending from Union College and Prof. Mukkai Kitware attended the annual IEEE VisWeek 2010 Conference Krishnamoorthy from RPI and Will Schroeder, Bill Hoffman, in Salt Lake City, UT in October. The VisWeek conference Niki Russell, Katie Cronen and Luis Ibáñez from Kitware.
16 RSNA 2010 interface allows for a more rapid navigation for the user. The This year the annual meeting of the Radiological Society of 3D imaging and segmentation capabilities of VolView are America (RSNA 2010) featured 60,000 attendees, 4,200 sci- among the most advanced in the world. entific expositions and more than 700 technical exhibitors “Kitware’s goal is to provide this advanced imaging tool to at McCormick Place in Chicago. It also featured the first researchers around the world and work with the imaging ever Kitware technical exhibition at Booth 1305, Lakeside community to improve the capabilities and help us make Center. RSNA offers a unique opportunity to meet with a future version of VolView system both free and open many of the researchers, collaborators and customers with source,” said Rick Avila. whom we work but do not normally get to see. It also pro- vides an excellent opportunity to meet new collaborators and customers. The addition of a Kitware booth gave us a central location and allowed us to showcase more of the open-source solutions that we help support. Along with the technical exhibit, Kitware, maintained a high level of participation in the conference. Stephen Aylward and Rick Avila reprised their wildly successful “Open-Source Tools for Medical Research and Applications” course. The 90 minute session provided a quick introduction to ITK, VTK and Slicer 3D and was intended to get an interested audience started on the open source path. Rick Avila fol- lowed this up with some recent Kitware research into “The Evaluation of a New Pocket-sized CT Calibration Phantom” during the technical sessions. Kitware was also asked to participate in the Quantitative Imaging Reading Room and responded with “An Open Source Toolkit for Quantitative Lesion Sizing” focusing on the Lesion Sizing Toolkit (LSTK), which is soon to become a part of ITK.
Finally, as part of the spirit of free and open-source software, A contrast CT scan and the corresponding image in VolView Kitware announced that it will no longer be charging a fee for its VolView visualization platform. Although not approved by the Federal Drug Administration (FDA), hence restricted from use in clinical settings, VolView has immense research value and real-world medical poten- tial. For instance, a 3D rendering of the size and shape of an ailing patient’s aorta would allow medical researchers to evaluate whether certain display methods are better than others for identifying subtle aneurisms. Kitware works col- laboratively on research endeavors like this and is actively seeking funding to move VolView from a free tool available in binary format to a true open-source platform. ACTIVIZ IS NOW FREE AND OPEN-SOURCE The source code for ActiViz.NET and mummy, Kitware’s tool for generating C# wrappers around C++ classes, is now avail- able as open source under a BSD license. ActiViz.NET is a tool for generating C# wrappers around VTK, an advanced open-source software system for visualization. VTK is an object-oriented software system for 3D graph- ics, image processing, data visualization, data processing, volume rendering and more. VTK supports a wide variety Kitware's Karthik Krishnan, Julien Jomier and Wes Turner of algorithms, such as scalar, vector, tensor, texture; and at the Kitware booth in McCormick Place. advanced modeling techniques such as: implicit modeling, polygon reduction, mesh smoothing, cutting, contouring, and Delaunay triangulation. KITWARE OFFERS FREE ACCESS TO VOLVIEW VTK, written in C++, does not easily integrate into the In Novemeber, Kitware announced that VolView, an intui- Microsoft development environment. ActiViz.NET was tive and interactive system for volume visualization, is now developed to overcome this issue by building an integration available free of charge to researchers around the world. layer for VTK, enabling use in the Microsoft .NET frame- Compatible with Windows, Mac or Linux, VolView is capable work. Originally a proprietary product, ActiViz.NET is now of volume rendering, maximum intensity projections and open-source, allowing developers the freedom to customize oblique reformatting. In addition, VolView can load multiple and utilize the software to meet their specific needs. While datasets simultaneously, support creating and applying 2D ActiViz.NET comes with an application to be used with the and 3D display presets and even make image measurements Wikipedia Browser, it requires writing software to take full within a user defined contour. This simple and easy-to-use advantage of the functionality of VTK.
17 NEW OFFERINGS FOR OPEN SOURCE COURSES Troy, NY. Brad continued working at Kitware while complet- Kitware is pleased to announce two new options for train- ing his undergraduate and graduate education, receiving his ing courses. We will be hosting our first course in Europe Ph.D. in December 2008. Dr. King is an invaluable member February 2-4, 2011 in Lyon, France. This course will cover of the Kitware software development team, and is a lead the Insight Segmentation and Registration Toolkit (ITK), contributor to CMake, Kitware's most popular open-source the Visualization Toolkit (VTK), CMake and ParaView, and project. Brad is also a major contributor to VTK's advanced is suitable for developers who would like to learn more pipeline architecture and the ITK wrapping process. about these tools and for project managers looking to inte- Berk Geveci joined Kitware in September 2000 after com- grate these software tools into their team’s development pleting his Ph.D. at Lehigh University, and a post-doctoral environment. The course will be a hands-on experience position at the University of Pennsylvania. Dr. Geveci has and attendees can work in the development environment been instrumental in growing Kitware's scientific visualiza- of their choice (Windows, Linux or Mac), allowing them to tion business, and now serves as the Director of Scientific better focus on the content. Computing for Kitware. Berk is one of the lead developers Additionally, Kitware is debuting new online courses. of both ParaView and VTK, and has extensive expertise in There will be various topics covered on the different tool- the areas of large scale parallel computing, computational kits, including ITK, VTK, CMake and ParaView. A variety of dynamics, finite elements and visualization algorithms. His instructor-led 90-minute online courses will be offered, cov- award-winning efforts have resulted in Kitware's tools being ering both introductory and more advanced topics in each used in numerous government and commercial research open source project. labs, and on most of the world's top high-performance com- puting platforms. For a complete course catalog with detailed descriptions, course calendar and registration information, please visit OPEN SOURCING MICROSCOPY: CSHL MEETING the training page on our website, www.kitware.com. If In December, Luis Ibáñez and Dhanannjay Deo attended you would like to arrange a custom course tailored to your the Cold Spring Harbor Lab meeting on High Throughput precise needs, please email [email protected] or call us Image Analysis in Phenotyping. Phenotyping is the study at (518) 371-3971. of observable characteristics of an organism resulting from the expression of its genes and their interaction with the KITWARE HONORS DEDICATED EMPLOYEES environment. The meeting attendees discussed their work In December, Kitware presented Brad King and Berk Geveci in image analysis in phenotyping, some of the challenges with awards commemorating their 10 years at Kitware, a and obstacles they face in doing their research and possible career milestone that speaks to their strong commitment methods for improvement. to both Kitware and the greater open source development community. One of the biggest challenges facing biologists in this field is the massive amount of data produced; it is now common to run experiments whose output amounts to several terabytes of data. Each researcher has several experiments in progress simultaneously and the need for storage and advanced image processing capabilities for this data is of the utmost importance.
Cold Spring Harbor Lab, where Watson and Crick discovered the double-helix molecular structure of DNA.
Biologists have generally recognized the value of open source Berk Geveci and Brad King honored for 10 years of dedication in research and helped usher in the open access movement. As such, the discussion on moving the field of microscopy Brad King joined Kitware in June 2000 as a summer intern image analysis forward included several references to open while pursuing his B.S. at Rensselaer Polytechnic Institute in source solutions. The key topics included archiving for
18 image storage, standard data formats and interoperability NA-MIC Project Week for data sharing, knowledge representation, interfaces and January 10-14 in Salt Lake City, Utah. The project week making open source resources interoperable, and funding provides hands-on R&D for image-guided therapy and neu- issues. With open source as a feature of several of these roscience applications. Stephen Aylward, Will Schroeder, topics, appropriate funding is necessary to develop high- Julien Finet, Jean-Christophe Fillion-Robin, Zach Mullen, quality maintainable software and an issue that needs to be Danielle Pace and Dave Partyka will be in attendance. addressed by funding agencies such as the NSF and NIH. http://www.na-mic.org/Wiki/index.php/2011_ The talk was attended by representatives from top interna- Winter_Project_Week tional labs and the developers of OME, ImageJ, Bio-formats IS&T/SPIE Electronic Imaging and CellProfiler. The meeting was fruitful and some very January 23-27 in San Francisco, California. Arslan Basharat, interesting projects, such as spectral plant imaging and the Amitha Perera and Wes Turner will be presenting a poster use of micro-fluidics in C.Elegans worm phenotyping, were “Tracking flow of leukocytes in blood for drug analysis.” covered. There has been a follow-up meeting scheduled for http://spie.org/x16218.xml April 2012. SPIE Medical Imaging BOOKS NOW AVAILABLE ON AMAZON UK February 12-17 in Lake Buena Vista, Florida. SPIE is the To continue to better serve our European customers, Kitware premier conference for medical scientists, physicists, and prac- is pleased to announce the availability of Mastering CMake titioners in the field of imaging. Stephen Aylward is actively and The VTK User's Guide on www.amazon.co.uk. This involved in this year’s conference, serving as a program enables our European customers to obtain the books with committee member for the Computer-Aided Diagnosis less expensive and more flexible shipping options. (CAD) conference and a co-chair for the CAD workshop. See Mastering CMake is now in its fifth edition and covers CMake, the CAD workshop website for the call for participation. the extensible open-source system that manages build pro- http://www.kitware.com/workshops/SPIE_CAD_2011.html cesses in an operating system and compiler independent IGSTK User Meeting manner. The VTK User’s Guide is in its eleventh edition and February 12, 2011 in Orlando, FL. This user meeting is an covers the Visualization Toolkit, an open-source, C++ toolkit opportunity for collaborators and users to gather and discuss for scientific visualization, informatics and 2D and 3D data their research projects and experiences. Andinet Enquobahrie processing. will be in attendance. http://public.kitware.com/IGSTKWIKI/ We plan to extend the current offerings by adding more index.php/IGSTK_User_Group_Meeting%2C_SPIE_2011 titles and additional Amazon sites in the coming year. NA-MIC Registration Retreat February 19-23 in San Juan, Puerto Rico. The retreat brings together registration algorithm researchers from NAC, NA-MIC, NCIGT and other organizations, including Kitware. Stephen Aylward, Danielle Pace and Will Schroeder will be in attendance. http://www.na-mic.org/Wiki/index.php/ Event:2011_Registration_Retreat VIZBI 2011 March 16-19 in Cambridge, Massachusetts. Jeff Baumes and Wes Turner will be presenting a tutorial on the funda- mentals of VTK and ParaView and how they can be used to construct visualization pipelines used in biological research. http://vizbi.org/2011/
NEW HIRES Laura Curthoys Laura joined Kitware as an Office Assistant in October. Prior to joining Kitware, Laura worked extensively in operations and administrative support positions. Michael Grauer Michael joined Kitware as a developer for Digital Libraries and Informatics in our North Carolina office in November. He UPCOMING WINTER CONFERENCES AND EVENTS received his B.A. in computer science and history from the If you’re interested in meeting with a Kitware representative University of Texas at Austin and his M.S. in computer science at one of these events, email us at [email protected]. from Drexel University. Prior to coming to Kitware, Michael worked as a bioinformatics specialist at the Children’s IEEE Winter Vision Meeting Hospital of Pennsylvania. January 5-7 in Kona, Hawaii. The meeting is a set of co- located workshops: Applications of Computer Vision, Deb Howell Motion and Video Computing and Person-Oriented Deb joined Kitware as the Office Manager for the Chapel Vision. Kitware’s Dr. Anthony Hoogs is a program co-chair Hill office. Prior to joining, Deb was the office manager for for the Workshop on Applications of Computer Vision. a security company. She graduated from the Architectural http://vision.cs.byu.edu/wvm2011/wvm.php Engineering program at Greenville Technical College.
19 Charles Marion Interns are valuable members of their respective teams Charles originally joined Kitware in 2008 as an intern in the and have sometimes continued to contribute to Kitware as Chapel Hill office and is now an R&D engineer. He is working employees after the completion of their internship. One of in Kitware’s office in Lyon, France, where he is involved in Kitware's early employees, Brad King, started as an intern projects in medical imaging, computer vision and data pub- and recently celebrated 10 years at Kitware. Nikhil Shetty lication. Charles received his Master’s degree in electrical and Charles Marion also started as interns and are now engineering and information processing from the University members of the research staff. of Lyon. If you are interested in applying for a Kitware Internship, Katie Osterdahl please send your resume to [email protected]. Katie joined Kitware as the Communications Specialist in October. Prior to joining Kitware, Katie worked as the EMPLOYMENT OPPORTUNITIES marketing and communications assistant for an insurance Kitware is seeking talented, motivated and creative indi- company in London, England. She received her B.S. in viduals to become part of our team. As one of the fastest Advertising from Syracuse University in 2008. growing companies in the country, we have an immedi- ate need for software developers, especially those with Vicki Rafferty experience in computer vision, scientific computing and bio- Vicki Rafferty joined Kitware in December as the new medical imaging. Contracts Administrator. Prior to joining, Vicki worked for RPI in their Research Administration and Finance depart- At Kitware, you will work on cutting-edge research prob- ment, most recently as the grant administrator. lems alongside experts in the fields of visualization, medical imaging, computer vision, 3D data publishing and technical Linus Sherrill software development. Our open source business model Linus joined Kitware in October as an R&D Engineer for means that your impact goes far beyond Kitware as you the Computer Vision team. He received his B.S. in computer become part of the worldwide communities surrounding science from the State University at Albany and M.S. in com- our projects. puter science from Union Graduate College. Prior to joining Kitware, Linus worked at Infoscitex as a software architect Kitware employees are passionate and dedicated to inno- and developer. vative open-source solutions. They enjoy a collaborative work environment that empowers them to pursue new KITWARE INTERNSHIPS opportunities and challenge the status quo with new ideas. Kitware Internships provide current college students with In addition to providing an excellent workplace, we offer the opportunity to gain hands-on experience working with comprehensive benefits including: flexible hours; six weeks leaders in their fields on cutting-edge problems. Our busi- paid time off; a computer hardware budget; 401(k); health, ness model is based on open source software—an exciting, vision, dental and life insurance; short- and long-term dis- rewarding work environment. ability, visa processing; a generous compensation plan; profit sharing; and free drinks and snacks. Kitware Internships can range in duration, from a summer or winter break to a year, enabling the intern Interested applicants are encouraged to send their cover to delve into a project and make a contribution toward a letter and resume to [email protected] to ensure their real-world solution. immediate consideration.
In addition to providing readers with updates on Kitware The Source is published quarterly by Kitware, Inc., Clifton product development and news pertinent to the open Park, New York. source community, the Source delivers basic information on Contributors: Lisa Avila, Utkarsh Ayachit, Jeff Baumes, Dave recent releases, upcoming changes and detailed technical Cole, Katie Cronen, Dave DeMarle, Arnaud Gelas, Ghassan articles related to Kitware’s open-source projects, including: Hamarneh, Marcus Hanwell, Zhi Feng Huang, Luis Ibáñez, • The Visualization Toolkit (www.vtk.org) Julien Jomier, Wen Li, Matt McCormick, Vincent Magnotta, • The Insight Segmentation and Registration Toolkit (www.itk.org) Sean Megason, Kishore Mosaliganti, Dave Partyka, Antonin Perrot-Audet, Cory Quammen, Nicolas Rannou, Will Schroeder, • ParaView (www.paraview.org) Boris Shabash, Lydie Souhait, Russell Taylor, and Wes Turner. • The Image Guided Surgery Toolkit (www.igstk.org) Graphic Design: Steve Jordan • CMake (www.cmake.org) Editor: Katie Osterdahl • CDash (www.cdash.org) • MIDAS (www.kitware.com/midas) Copyright 2010 by Kitware, Inc. or original authors. • BatchMake (www.batchmake.org) The material in this newsletter may be reproduced and distributed in whole, without permission, provided the above copyright is kept Kitware would like to encourage our active developer intact. All other use requires the express permission of Kitware, Inc. community to contribute to the Source. Contributions may Kitware, ParaView, and VolView are registered trademarks of Kitware, include a technical article describing an enhancement you’ve Inc. All other trademarks are property of their respective owners. made to a Kitware open-source project or successes/lessons learned via developing a product built upon one or more To contribute to Kitware’s open-source dialogue in future of Kitware’s open-source projects. Authors of any accepted editions, or for more information on contributing to specific article will receive a free, five volume set of Kitware books. projects, please contact the editor at [email protected].
20