Utilising a commercial workflow when reconstructing dinosaur skeletons in 3D

Reconstructing fossils digitally has become a vital technique in palaeontology in recent years. The ability to reconstruct the entire skeletons of extinct taxa gives us the opportunity to look at aspects of an organism’s life that would prove difficult or impossible using physical models, for example in biomechanics and gait analysis.

Er, hi. The final, posed Gorgosaurus model, created in zBrush 4R7 and Cinema 4D R16.

A recent paper by Stephan Lautenschlager  (http://rsos.royalsocietypublishing.org/content/3/10/160342) on digital fossil restoration techniques provides a good overview of the methodology using tools available for digital palaeontology in most labs, including Avizo, Blender and Maya. I highly recommend this paper for anyone interested in the digital restoration of fossil specimens and the workflow involved, much contained in the paper being relevant to the digital reconstruction of specimens for research and outreach.

Last year I worked on a project with Dr. Willam Sellers of Animal Simulation Laboratory (http://www.animalsimulation.org) at the University of Manchester which required the creation of six accurate and complete dinosaur skeletons (see my previous post on the project: http://paleoillustrata.blogspot.co.uk/2016/01/walking-and-making-dinosaur.html), and for this I used a workflow developed in my work creating scientific animations and illustrations for the MedComms industry, a workflow not dissimilar to many commercial studios where 3D assets of all description are produced. axon, the makers of one of the 3D modelling and animation applications I use, Cinema 4D R18, covered this project on their website: https://www.maxon.net/en/industries/visualization/article/everybody-walk-the-dinosaur/

A specimen (NHM R175) recorded using photogrammetry
and suitable for use as a basis for reconstruction of the bone.

The reference used to create individual elements comes from a number of sources. Primary amongst these are meshes generated using with photogrammetry, Lidar or CT scanning. These data are often received as point clouds that require the generation of a mesh. These datasets can be pretty hefty (especially if they are of entire mounted skeletons or other large, complex subjects) so they require chopping down into manageable chunks prior to meshing to make processing more efficient, for example isolating part of a limb or the skull. I use MeshLab (http://meshlab.sourceforge.net) for this part of the processes, saving sections of the point cloud and meshing them individually. The meshes generated from this procedure are the basis for the reconstruction and these are imported as reference meshes into the main 3D package prior to starting modelling. The literature was consulted and where possible collections were visited and photographs taken to maximise the available reference for modelling. This is important because as there is a fair amount of morphological variation within a particular taxa and this needs to be taken into account when modelling the skeletons, and in fact the final models are composites and don’t represent a single specimen.

Like many studios I use more than one application to create my models of palaeontological specimens. As mentioned earlier, the main 3D application I use for both research and commercial work is Cinema 4D R18 by Maxon (https://www.maxon.net). Within C4D I block out the basic form of the model by generating primitives, extruding polygons and dragging edges and points into approximate position (see this post for how this works: http://paleoillustrata.blogspot.co.uk/2011/09/building-dinosaur-pulling-polygons-and.html). To make these models practical for biomechanical work and 3D printing, so they needed to have as few polygons as possible whilst retaining accuracy. By starting with very simple, blocky shapes it is far easier to keep control of the polygon count as you model and make sure the mesh density is as low as possible for the final model.

Low resolution, blocked-out skull ready for importing to zBrush for further sculpting (top),
 and the skull in the process of being modelled at a higher resolution.

Once the basic shape is modelled in C4D, the meshes (both reference and model are imported into Pixologic zBrush 4R7 (http://pixologic.com), a 3D modelling package that uses sculpting in clay as a metaphor for modelling 3D meshes. Why not continue modelling in Cinema 4D or anther package? Well, zBrush’s tools are intuitive, fast and easy to use (once you’re used to them) and well suited to creating organic shapes. As a modelling tool widely used in the gaming industry zBrush also offers excellent retopologising and mesh decimation tools, which means models can have a low mesh density whilst retaining detail; it produces very ‘clean’ meshes which are perfect for importing into a wide variety of other applications. Once the model is completed the final mesh is imported back into Cinema 4D for positioning and texturing (if required). I cannot recommend zBrush highly enough for organic modelling and anyone creating reconstructions in 3D should take a close look at this package.

The skull of Gorgosaurus during the modelling process, showing the layout within zBrush 4R7.

Assembling a skeleton in Cinema 4D is a task made easier by some of the tools that are add-ons to the application and widely used by modellers and animators. Foremost amongst these is the Mograph module, which allows the cloning of a single element and the manipulation of these clones. This is tremendously useful when creating multiple copies of broadly similar elements such as vertebrae. For example, the tail several vertebrae from certain points along the sequence would be modelled and then those in between generated by cloning and these would then be adjusted individually to match the reference.

The Triceratops skeleton rigged and hot to trot in GaitSym.

The final model can be exported out as any type of 3D file, with the most useful being .obj and .fbx at present, but anticipate this changing in the future and save in several formats as well as the proprietary format of the modelling applications.

The Giraffititan model, posed.

The use of commercial tools such as zBrush is becoming more common for palaeontologists and palaeoartists, and by paying close attention to the structure of production pipelines and workflows of commercial digital studios is something we as palaeontologists should consider, as there is much to learn from their techniques and methods which we can incorporate into our own reconstruction and restoration arsenal of techniques for generating 3D meshes useful or wide variety of research pathways.

Fossils on the small screen at Dinosaur Encounter, SeaCity.

As part of the exhibition at SeaCity I was commissioned to create videos to accompany two of the exhibits, the Sandown Academy crocodile which was worked on by Michela Johnson and the nodosaurid which is being worked on by myself and Sarah Strachan. These videos are intended as interpretive aids to help visits understand the fossils and hopefully allow people to see how the bones fit together and give an idea of what the animals looked like in real life.

Still from the croc video showing bone outline highlighted to
aid interpretation of the specimen.

The croc skull itself is still encased in the block of matrix it was found in. This means that from the outside only cross sections of bone can be seen and these can be difficult to interpret to the untrained eye (and often to the trained eye too). The skull has been scanned using the University of Southampton’s CT scanner and 3D data had been extracted. The croc skull video features footage of this CT data and concentrates on how this technique allows us obtain detailed data without even prepping the fossil. In fact, as CT resolution improves over the years it’s entirely possible we won’t need to prep some specimens that might prove difficult to reveal for any number of reasons including matrix that is too hard or too soft. The great thing about the CT data is it translates readily into visuals and thus lends itself to motion graphics; we can highlight certain elements and add labels to aid understanding. 

The crocodile video next to the specimen as part of the display.
Image: Liz Martin.

The Polacanthus video is more focussed on the bones themselves. As I’m working on this specimen for my PhD I already had enough data to attempt a reconstruction for the video, which is played on a screen situated above the display case containing the dinosaur. It has to be stressed this reconstruction was the first I have done of the dinosaur, and was produced primarily for the video and not publication. For the sequences showing the bones of the specimen I used the actual photogrammetry data and this also enabled me to make relatively accurate inferences about the length and height of the animal. I based the missing elements of the skeletal reconstruction on Polacanthus foxii and other nodosaurids. The final muscle and life reconstructions were far more speculative but give a relatively good idea of how the animal looked in life. The neutral pose was chosen deliberately to keep the orientation of the bones as easy to see on screen as possible; unfortunately time did not permit a more dynamic reconstruction, but watch this space . . . 

First reconstruction of the nodosaur currently being studied at
University of Southampton. Video still.

Scientific illustration: Cell schematic

Generic human cell illustration. Click to embiggen.

I've recently been working on a piece that I thought might be of interest to readers of this blog as although it's not palaeontology, who can resist a bit of biology?

This is a 3D model created in entirely in Maxon's excellent Cinema4D of a cell that is gracing the front page of my day job website (www.stupond.com), and is a schematic of a generic human cell showing the basic structures and organelles commonly present in many cells. This could be animated or labelled is needed.

Next on the personal learning curve is getting to grips with Maya, one of the real heavy-hitters in the world of 3D modelling with a learning curve to match. Exciting stuff!

I hope you like the illustration.

Building a dinosaur: Starting to detail Triceratops

The next stage in the construction of our Triceratops model is one on the fun parts of creating 3D models: detailing. This is mainly carried out in zBrush and takes full advantage of the symmetry option which allows the artist to mirror brush stokes over the axis of choice. This means you can add detail without having to do it twice and as dinosaurs are symmetrical this is extremely useful. Of course, there will be subtle asymmetrical details that will need to be added but these can wait until we have the main details sculpted.

As with the rest of the modelling process, detailing is best approached by starting out with the larger details first and working in ever-increasing resolution as the finer details are added. There are decisions to be made at this stage too, deciding how much to actually model with geometry and how much to add using displacement maps and texturing. For the time being, I'm going to continue adding to the geometry as we're still dealing with larger details rather than the tiny stuff.

As you can see I've started to refine some areas that were difficult to model by pulling polygons and points, such as much of the skull detailing including the rostral and orbit areas, the epioccipitals and general refinements around the skull. I'm also adding wrinkles to the skin, especially in limb areas. I'm also attempting to keep the animal looking quite fleshy as I'm anxious too avoid an overly skinny look with bones poking through everywhere. Most of this work is done using standard zBrush brushes and alphas; the need for custom alphas will be when the finer detailing such as skin texture is being applied.

Many of the decisions on how the muscles look are based on our earlier reconstruction of the musculature of Triceratops, but at this stage it's worth considering how the integument of the animal affects the way the skin might fold and fall across the skin and muscles. Keep referring to any reference you have gathered and be mindful of the analogues you choose; elephant skin is probably very different to ceratopsian skin, so do the research. More on this in the next post.

Confuciusornis - gathering 3D data from a museum exhibit

A couple of weeks ago I took a trip up to Manchester Museum to peruse the excellent fossil gallery and test out my new camera (a Fuji x10). Whilst there my eye was caught by a rather fine fossil of the Lower Cretaceous bird Confuciusornis sanctus, from the Jianshangou Bed of the Yixian Formation (the label in the cabinet gives the location as the Chaomidianzi Formation but according to Wikipedia[!] this is a synonym for the Jianshangou Bed and has fallen out of use; the ref citied in the wiki is behind a paywall and so I can't access it to check), West Laioning Province, China. I thought it might be worth trying to gather 3D data from the fossil using the technique of photogrammetry. Although the specimen is behind glass I got good data, and here are the results.

The first image is a conventional photograph taken from directly in front of the fossil. I've adjusted the levels in the image to bring out the detail. Note the rather wonderful feather impressions of the wings and the fuzz on the neck.

This is the 3D data generated from the photographs taken for reconstruction using photogrammetry. There were 38 images in all, and these were combined to generate the above mesh. The bones are clearly visible, and the feathers of the left (our right, we're looking at the front of the bird) wing are also visible. The rectilinear features visible on this mesh are possibly artefacts of the alignment of the photos, perhaps due to subtle changes in the way each photo was lit or having the glass in-between the camera and the subject. However, I'd have to check this - oh for the luxury of on-site processing!

Above is an unadjusted render of the final textured mesh. Below is a close-up of the final mesh, with the camera positioned at a jaunty angle which shows off the quality of the model. This is lit with a single omni light with ambient occlusion active.

The quality of the mesh is well displayed in this image, and enables us to examine the fossil from all angles and also by loading the model into a viewer we can take a high-quality 3D model of this fossil anywhere we can take our laptops, iPads or smartphones. This is an effective way of accessing fossils, gathering usable data which is readily available at any time, and creating a record of the specimen which is useful for research.

Planet Dinosaur producer interview

The BBC website has a short interview with Nigel Paterson, producer of Planet Dinosaur talking about the science behind the programme. Nothing revelatory here but a chance to get a glimpse of the production process and the work done by the 3D artists.

Building a dinosaur - pulling polygons and symmetry

You'll recall from the last installment of this occasional series that before starting any actual modelling we needed to do the research. With that done, we can now start modelling Triceratops in earnest.

I use Cinema 4D for all my 3D work but the techniques I use are common to most major 3D modellers, with one notable exception, the mighty ZBrush which we'll cover later in the series. How you set up your production pipeline is personal to each individual and company, but mine is as follows:

Basic Modelling: Cinema 4D

Refinement and adding detail to model: ZBrush

Texturing: Bodypaint, ZBrush, Photoshop

Rigging: Cinema 4D

Animation: Cinema 4D

Rendering: (you guessed it!) Cinema 4D Advanced Renderer

Post production and compositing: Adobe After Effects

Before starting modelling I set up Cinema 4D (c4d) so I can model without having to stop and fiddle with lights etc along the way. So I create a basic matt pale grey texture, construct a standard 3-light setup (I actually have a master file I can import these elements from each time I start modelling) and finally add a floor for the model to stand on.

I then import any images I am going to use as reference for the model. This could be sketches, photos etc but in this case it's the skeletal Scott Hartman kindly gave me permission to use. I set this up in one of the views (in this case front) and use this as a guide to the proportions of my model.

Figure 1: The modelling environment; now all I need is a dinosaur to go in here . . .

Now we can start modelling. Luckily for us 3D modellers tetrapods display bilateral symmetry, which means they are they have the same layout on either side of the sagittal plane (see fig. 2). We too can use the sagittal plane as the axis for generating the mesh that will make up our model.

Human anatomical planes. As humans and dinosaurs are both tetrapods we can use

this scheme to assist in modelling our Triceratops. From here and used under

Creative Commons Licence.

To do this I use a symmetry object in c4d. This means any geometry created or modified is reflected along the chosen axis to create a whole model. By positioning a cube primitive with one side along this plane we can model one side of the dinosaur and the other side will be created automatically. Figures 3 and 4 show how this works.

Figure 3: A cube primitive aligned along the X-axis in c4d. The polygon against the axis

has been removed to make sure hypernurbs work correctly.

Figure 4: Drop the cube into a symmetry object, adjust the mirror plane to XY and viola!

Finally, the symmetry object containing the cube is dropped into a hypernurbs object, which automatically subdivides the geometry interactively to create organic forms; points and polygons can be weighted to adjust the influence the hypernurbs object has on the mesh.

Figure 5: The basic cube shifted so one edge aligns with the XY axis, dropped in a symmetry object

which is then dropped into a hypernurbs object.

Figure 6: The hierarchy of the basic starting mesh.

The hypernurbs and symmetry object can be turned off and on to help with modelling and previewing and you'll find yourself doing this constantly as the model progresses.

The technique I use for modelling is very simple. Extrude a polygon, adjust the points, extrude another and so on. Starting from this five-sided cube (remember we deleted the poly that is flush to the sagittal plane) we can create any shape we want. In the case of virtually all models including our dinosaur we want to rough out the body shape before we start getting to involved in detail. This basic rule of thumb, start simple and refine down gradually, and this needs some careful forward planning. Our dinosaur is basically one large, complicated shape and our mesh will end up as one large, complicated shape too. So we don't get lost, we need to pay close attention to the anatomy of our subject. We need to allow for the head and it's appendages, the limbs and their morphology and such details as the number of toes, the shape of the tail and so on. Where do the legs join onto the body? How does the neck articulate? How does the tail move? Think of these things before you start and the whole mesh as being divided down into ever smaller boxes and that's pretty much how the modelling process for this stage works.

Figure 6: Roughing out the body shape. This is the very basis of the model. I've extruded more

polygons from the original cube we started with and have begun to form the shape of the head,

body and tail. You can see I've not adjusted the Z co-ordinates at all, at this stage I'm only interested in

roughing out the form along the X and Y axes. The highlighted polygon will be where the shoulder

and front leg of the animal will be modelled.

So that's the start. Check out online tutorials for your specific modelling application for other techniques that will be of use as you progress, but remember the mesh has to be a single object, preferably made of polygons. Be thrifty in your use of polygons as we want to keep the polygon count as low as possible as they will soon mount up, and if we want to animate then lots of polygons could be problematic.

One more thing - don't move the points on any of the polygons on the X/Y axis (the sagittal plane) of the model, as this will create gaps in the symmetry later on, and you final model will have holes. Keep checking everything's in order by flicking on the symmetry and hypernurbs as you work, and correct any errors as you go along - do not leave them for later or you might get in one awful muddle.