Project funded in 2008 through the program PROFIT (Tractor project) by the Spanish Ministry of Industry, Tourism and Commerce.
Development of a software application to reconstruct individual faces starting from their skull in legal and forensic medicine.
Human identification from a skull is a critical process in legal and forensic medicine, especially when no other means are available. Traditional clay-based methods attempt to generate the human face, in order to identify the corresponding person. However, these reconstructions lack of objectivity and consistence, since they depend on the practitioner. Moreover, the results of the reconstruction cannot be easily distributed and consulted from everywhere.
The main objective of this project is to develop a 3D craniofacial reconstruction automatic system. This system consists of a main application able to generate a 3D mesh simulating the skin of a person from the 3D image corresponding to his/her skull. Facial features are not included in the generated mesh, as they cannot be confidently deduced only from skull. For this purpose, a different, but completely objective, method is used. Both input data and generated reconstructions have been stored in a database that could be accessed through a suitable web application. The application, acting as a medical documentation repository, is intended for professional experts only, that should be registered to the service.
Another objective of the project CRÁNEO is the development of a software that automates the process of skull-photo superimposition used by forensic experts in identifying character. This software will automatically superimpose the 3D geometry of the scanned skull with any 2D photograph regardless of the angle it was taken. This way it is possible to generate a large number of overlays in a rapid and automatic way.
Soft tissue generation process
3D FACIAL RECONSTRUCTION. The main application scheme comprises the elements in the Figure below:
1. Main application scheme
- Input Data: Set of data to generate the soft tissue mesh. Three different input data are required:
- Skull image: a 3D image of the skull needed to obtain its facial reconstruction;
- 66 landmark points placed on the skull surface: at those reference points, soft tissue depth is known.
- Age, gender and BMI range of the person the skull belongs to. Age and gender can be deduced from skull morphology, so they will always be known by the user. However, BMI range will be an unknown parameter, so it will be estimated.
- Landmark insertion module (LIM): This module is in charge of placing the 66 reference points on the skull surface and assigning them a tissue depth value, according to age, gender and BMI parameters. The reference points used for this purpose are two sets of points traditionally used in forensic medicine, as shown below:
2. Landmarks used for craniofacial reconstruction in the project: a first set of 52 points to generate the reconstruction in facial zone (black) and a second set of 14 points to generate craniofacial reconstruction in neurocranium (red)
- Soft tissue generation module: The Skin Mesh Generation Module (SMGM) block represents the main functional module in the application. Based on the LIM output data, it generates a set of intermediate points on the skull surface, whose depth values can be interpolated from thickness values in reference points (see Figure 3). The whole set of points (landmarks and intermediate points) are used to build the final 3D mesh representing the soft tissue (skin) corresponding to the given skull.
3. Soft tissue generation process
SKULL-PHOTO SUPERIMPOSITION. The main application scheme is represented below:
4. Skull-Photo Superimposition scheme
- Input Data: Set of data used by the application to generate the soft tissue mesh. Three different input data are required:
- Skull image: a 3D image of the skull needed to obtain its facial reconstruction.
- 5 landmark points placed on the skull surface.
- 5 cefalometric points placed on 2D photo.
- Facial Features Detection Module: this module automatically detects face, eyes and nose on the photography. These positions will be useful to calculate cefalometric points correctly
5. Detection of face, eyes and nose areas on the photo.
- 3D Points Insertion Module: This module maps the set of 5 reference points on the 3D skull. For this end, the module developed for 3D Facial Reconstruction has been used (see Landmark insertion module).
- 2D Points Insertion Module: The 2D Points Insertion Module overlays the set of 5 reference points on the photograph
6. Example of photo with final points calculated.
- Homography Calculation Module: After estimating the position of the points on the skull and on the photograph, it is necessary to calculate the correlation between them. The correlation between the two set of points is performed using a homography, that is a special projective transformation which maps the coordinate system x =[x1, x2] to x’ =[x’1, x’2] (see [Hartley and Zisseman (2000), Hartley and Zisseman (2003)]. The figure below visually represents this concept:
7. Example of homography between 2 sets of points
- Matching Module: The Matching Module calculates the degree of correspondence between skull and photograph points after they have been correlated by the homography.
ONLINE MEDICAL DOCUMENTATION CENTRE
A web environment for the 3D reconstruction application has been implemented to allow the remote visualization of both the input data and generated reconstructions obtained by the main application.
The interaction between remote (web) users and the main application is illustrated in the following figure. Through this scheme a user can visualize and modify any information contained in the application database (skull information and generated reconstructions).
8. Interaction between main application and web users
9. Video demo of reconstruction application