Utilising a commercial workflow when reconstructing dinosaur skeletons in 3D

Reconstructing fossils digitally has become a vital technique in palaeontology in recent years. The ability to reconstruct the entire skeletons of extinct taxa gives us the opportunity to look at aspects of an organism’s life that would prove difficult or impossible using physical models, for example in biomechanics and gait analysis.

Er, hi. The final, posed Gorgosaurus model, created in zBrush 4R7 and Cinema 4D R16.

A recent paper by Stephan Lautenschlager  (http://rsos.royalsocietypublishing.org/content/3/10/160342) on digital fossil restoration techniques provides a good overview of the methodology using tools available for digital palaeontology in most labs, including Avizo, Blender and Maya. I highly recommend this paper for anyone interested in the digital restoration of fossil specimens and the workflow involved, much contained in the paper being relevant to the digital reconstruction of specimens for research and outreach.

Last year I worked on a project with Dr. Willam Sellers of Animal Simulation Laboratory (http://www.animalsimulation.org) at the University of Manchester which required the creation of six accurate and complete dinosaur skeletons (see my previous post on the project: http://paleoillustrata.blogspot.co.uk/2016/01/walking-and-making-dinosaur.html), and for this I used a workflow developed in my work creating scientific animations and illustrations for the MedComms industry, a workflow not dissimilar to many commercial studios where 3D assets of all description are produced. axon, the makers of one of the 3D modelling and animation applications I use, Cinema 4D R18, covered this project on their website: https://www.maxon.net/en/industries/visualization/article/everybody-walk-the-dinosaur/

A specimen (NHM R175) recorded using photogrammetry
and suitable for use as a basis for reconstruction of the bone.

The reference used to create individual elements comes from a number of sources. Primary amongst these are meshes generated using with photogrammetry, Lidar or CT scanning. These data are often received as point clouds that require the generation of a mesh. These datasets can be pretty hefty (especially if they are of entire mounted skeletons or other large, complex subjects) so they require chopping down into manageable chunks prior to meshing to make processing more efficient, for example isolating part of a limb or the skull. I use MeshLab (http://meshlab.sourceforge.net) for this part of the processes, saving sections of the point cloud and meshing them individually. The meshes generated from this procedure are the basis for the reconstruction and these are imported as reference meshes into the main 3D package prior to starting modelling. The literature was consulted and where possible collections were visited and photographs taken to maximise the available reference for modelling. This is important because as there is a fair amount of morphological variation within a particular taxa and this needs to be taken into account when modelling the skeletons, and in fact the final models are composites and don’t represent a single specimen.

Like many studios I use more than one application to create my models of palaeontological specimens. As mentioned earlier, the main 3D application I use for both research and commercial work is Cinema 4D R18 by Maxon (https://www.maxon.net). Within C4D I block out the basic form of the model by generating primitives, extruding polygons and dragging edges and points into approximate position (see this post for how this works: http://paleoillustrata.blogspot.co.uk/2011/09/building-dinosaur-pulling-polygons-and.html). To make these models practical for biomechanical work and 3D printing, so they needed to have as few polygons as possible whilst retaining accuracy. By starting with very simple, blocky shapes it is far easier to keep control of the polygon count as you model and make sure the mesh density is as low as possible for the final model.

Low resolution, blocked-out skull ready for importing to zBrush for further sculpting (top),
 and the skull in the process of being modelled at a higher resolution.

Once the basic shape is modelled in C4D, the meshes (both reference and model are imported into Pixologic zBrush 4R7 (http://pixologic.com), a 3D modelling package that uses sculpting in clay as a metaphor for modelling 3D meshes. Why not continue modelling in Cinema 4D or anther package? Well, zBrush’s tools are intuitive, fast and easy to use (once you’re used to them) and well suited to creating organic shapes. As a modelling tool widely used in the gaming industry zBrush also offers excellent retopologising and mesh decimation tools, which means models can have a low mesh density whilst retaining detail; it produces very ‘clean’ meshes which are perfect for importing into a wide variety of other applications. Once the model is completed the final mesh is imported back into Cinema 4D for positioning and texturing (if required). I cannot recommend zBrush highly enough for organic modelling and anyone creating reconstructions in 3D should take a close look at this package.

The skull of Gorgosaurus during the modelling process, showing the layout within zBrush 4R7.

Assembling a skeleton in Cinema 4D is a task made easier by some of the tools that are add-ons to the application and widely used by modellers and animators. Foremost amongst these is the Mograph module, which allows the cloning of a single element and the manipulation of these clones. This is tremendously useful when creating multiple copies of broadly similar elements such as vertebrae. For example, the tail several vertebrae from certain points along the sequence would be modelled and then those in between generated by cloning and these would then be adjusted individually to match the reference.

The Triceratops skeleton rigged and hot to trot in GaitSym.

The final model can be exported out as any type of 3D file, with the most useful being .obj and .fbx at present, but anticipate this changing in the future and save in several formats as well as the proprietary format of the modelling applications.

The Giraffititan model, posed.

The use of commercial tools such as zBrush is becoming more common for palaeontologists and palaeoartists, and by paying close attention to the structure of production pipelines and workflows of commercial digital studios is something we as palaeontologists should consider, as there is much to learn from their techniques and methods which we can incorporate into our own reconstruction and restoration arsenal of techniques for generating 3D meshes useful or wide variety of research pathways.

iPad art and the making of the mark

Technology is a wonderful thing, and occasionally a gadget comes on the market which is a real game-changer, and the latest of these to arrive at Paleo Illustrata is the iPad.  I've had it for around six months now and it's become a vital tool that makes doing art, work and science productive when away from the desktop. I'm typing this post on it now.

As an old-school graphic designer I've never really been that taken with the input devices available for drawing and painting on computers. I like pen and paper (see this post on sketch books for more). The mouse is fine for pointing and clicking, works great with bezier curves but makes a poor drawing tool. Tablets are better but still seem to make the drawing or painting process feel slightly detached from the artist. The iPad changes all this however. Combined with a stylus the iPad makes for an excellent painting and drawing too. Painting directly on the screen makes the experience seem very immediate, and it's positively pleasurable to be able to punish paint around on the screen. One of the device's leading advocates is the English painter David Hockney, who paints on his iPad and then prints the results out in large format.

Valley of Fire, Nevada. Painted by the author on an iPad2 using Procreate.

There are several painting and drawing apps on the market that are worth looking at. Brushes (£5.49) is Hockney's app of choice and indeed it is a very capable application, featuring layers and blending modes. Inspire pro (£5.49) is popular, again featuring layers and a wide range of brushes. My personal favourite is Procreate (69p at the moment - grab it now!), which allows HD size painting, includes layers and a wide range of brushes. However, Procreate is given a run for its money by the excellent Sketchbook Pro (£2.99) made by Autodesk. This app has a superb range of drawing and painting tools and is capable of high-quality results. Giant graphic sauropod Adobe have also weighed in recently with their tablet version of Photoshop, Adobe Photoshop Touch (£6.99). I haven’t tried this yet so can’t comment but hopefully the app avoids the bloatware behemoth syndrome of it’s desktop cousins.

3D modelling application 123d Sculpt (free) takes the iPad's tactile approach to interaction and allows the user to pull geometry around as if was clay. This approach works very well and whilst it's not in the same league as professional desktop 3D apps it is ideal as a sort of 3D doodler, and what's more is it's free.

There are many photography apps available from the AppStore, and in truth I tend to process images on my desktop in Photoshop. I mentioned earlier that Adobe has Photoshop Touch available, but I use Snapseed (£2.99) which has a range of effects and enhancement options, and is more than enough for tablet use, although I suspect that will change as the iPad develops.

Art is about making a mark. The marks an artist makes are how they define themselves, how they communicate. The iPad makes you feel like you are making a mark because of the immediacy of it's feedback; as you paint the colour appears under your stylus or finger. You can organize colour palettes quickly, get down big daubs of paint to block out composition and tonal range. It’s far from perfect, but for the first time a piece of technology has come along that complements the other tools I use in my research, art and day job.

Threads in deep time; reconnecting with who we are.

Scott Sampson, a dinosaur palaeontologist at Utah University is a man who thinks big. Very big in fact, as in his latest post over at his blog The Whirlpool of Life Scott discusses how we need to reconnect emotionally as well as intellectually with our place in the story of the evolution of not only life on our planet, but the evolution of the universe itself. He suggests we can each find our own thread that links us back to where we are and where we have come from in deep time and we can do this now, in the very locality we are at any one moment.

I picked up Scott's book Dinosaur Odyssey in Border's bookstore in Chicago last year, as Mrs Stu and I were on our way back to the UK after our week in the field in North Dakota and Montana. In the book, Scott uses dinosaurs as an example of how the flows of both information (evolution) and and energy ecology create into the web of life, how this web has changed from the almost unimaginable depths of deep time to the present day. The importance of food webs, the metabolism of dinosaurs and the role of technology is all discussed to illustrate this complex, remarkable and fascinating subject. It's a holistic view of dinosaur evolution and does of course, have real relevance for us as a species as the book demonstrates; it's also beautifully illustrated.

Scott's book struck a real chord with me because it was close to a subject I had been thinking about for a long time. Many years ago I was reading Cosmos by Carl Sagan, and (I've lost the book so this is a paraphrase) one passage really made an impression. Sagan suggests that if at the very least we are nothing more that biological machines, if our thoughts, memories and characters are nothing more than the firing of synapses and release of signalling proteins in the brain and if consciousness itself is explicable and ultimately quantifiable the at the very least we are the universe made conscious. We are the universe recognising itself and contemplating it's true nature (I think the great philosopher Alan Watts first suggested this idea but that quote eludes me too - sorry). This deeply profound insight has been with me ever since I came across it in Sagan's book.

See them galaxies? That's you, that is.

Interacting Galaxy Pair Arp 87. Image: NASA, ESA, and the Hubble Heritage Team (STScI/AURA)

To my mind the Sagan/Watts view provides answers to many questions. Firstly, it provides the basis for a firm moral framework that religions suggest atheist philosophy lacks, namely that every human life is unimaginably precious as it is the universe itself and this is such a profound fact that to cause intentional suffering to our fellow humans is a transgression from the natural order.

Next, if you take away this anthropocentric bias and apply this to all living things existing in their various states of consciousness, then we soon realise how precious and valuable everything that shares our planet with us is. The web of life Scott so brilliantly illustrates in his book is a part of us and we are a part of it, which is exactly what Scott suggests in his blog.

Like Scott Sampson I'm of the opinion we can and should reconnect emotionally with our sense of being and our sense of belonging in the natural order; concepts which science allows us to explore, give context to and perhaps one day understand fully. I think we all have that sense of something larger than us is at work in the universe; some people think it might be a creator or a supreme being (or several) and religion is a product of that recognition but for those of us for whom religion doesn't provide the answer how can we address this feeling? In it's search for truth science understands already that we are indeed part of something larger than us that we recognise through intuition and emotional connection as well as through intellectual inquiry. Books and blogs like Scott's and the work of other scientists and philosophers can give us a glimpse of the awe we feel when confronted with the grandeur and majesty of the natural world - from pulsars to protists, nebulae to nudibranchs, superstrings to sauropods.

A long-lost relative in the Lower Lias of North Yorkshire, UK.

But wait! This is a paleo art blog is it not? Well the arts are themselves an expression of our sense of being. The visual arts (such as paleo art) are as much a part of this broad and wonderful outpouring of human consciousness and creativity as anything else. As I argued in a previous post, it is desirable that some paleo art should transcend simply scientific representation and strive to express the essence of the life we are depicting, simply because we can. We are so connected to our subject (in our case palaeontology and ancient life) that we should allow ourselves to connect emotionally with what we are illustrating and in fact it might be essential to a degree; a part of our nature we cannot and should not suppress to much. My day job consists of depicting the internal mechanisms of the human body down to molecular level and this often requires a high degree of scientific accuracy, but it also leaves some room for creative licence within any given rationale (even the most utterly straight representation of data will require some interpretation by the artist). This is what I will be striving for on my journey into the art of palaeontology, my way of connecting with what Stephen J. Gould called "the old ones".

The Celts of these islands had a particular reverence for certain spots in the landscape such as springs, wells, rocks and trees and they believed (and many still do) these were inhabited by spirits particular to that specific location; the roman invaders called these spirits genius loci, the spirit of place. As Scott so eloquently argues, it is by discovering our own genius loci we can, through science and art reconnect with our own story and pick up our own very personal thread in the web of life and the universe.

Paleo Art or Scientific Illustration?

One of the most interesting issues I've come across in the paleoart blogsphere is the question of whether pictures of dinosaurs are art or science, and whether it's reasonable to illustrate dinosaurs in any way apart from using the latest science. Of course, accessing the latest science has become a bit of a hot topic recently and anyone following the lively and passionate discussions that followed Greg Paul's announcement that he wanted people to stop using his skeletal reconstructions will understand many people hold the subject of paleoart very close to their hearts.

But what about the actual function of palaeoart; purely scientific representation or could it be entertainment too? Could it go deeper than simply illustrating past life (not that this is actually a simple task) and illuminate some of the less quantifiable aspects of being a living being on our planet?


The image at the top of this post is actually from one of my sketchbooks. I had just acquired a rather natty brush pen and was experimenting with the strokes and assessing the quality of line, opacity of the ink etc. At some point I doodled this comic-book version of a large angry theropod, very loosely based on Tyrannosaurus rex and not ever meant to be seen by anyone. At the time, I was doing some pitch work for the BBC and had created a look I thought might work with and add something to this image. I scanned the sketch in, loaded it up in Photoshop and off I went. This was the result; a sort of deconstructionist take on dinosaur art with a nod to Frank Miller. I quite liked it, stuck it on my work website and got a positive reaction to the image from several people and so it stayed up. However, one thing always did bother me slightly about it - it simply wasn't accurate. It was never really meant to be, but did this matter? After all, I was more after the 'spirit of theropod'* than an actual reconstruction. 

Andrea Cau has argued in his excellent blog Theropda that the representation of dinosaurs in art should be purely scientific, that anything else is essentially a work of fantasy and imagination and should not be called paleoart. His superb The Decalogue of Paleoart lays out his view of what paleoart should be and for most of this I agree unreservedly; as paleo-artists our job is to communicate what we know about dinosaurs and other ancient life in a form that is easily understood and devoid of jargon so lay people can understand it (David Maas comments on the same post here and it's worth a read as Andrea clarifies some his points in the subsequent discussion). However, two of the Ten Commandments of Paleoart got me thinking:

VII - Do not create mythology 

VIII - Do not create false reconstruction

Hold on a moment - these two points are of course closely related in a sense. I can understand the mythology thing to a degree, perpetuating old myths and tropes (dragging sauropod tails etc) but am not so sure about "do not create false reconstruction"?  Even given the vagaries of Google Translate's sometimes eccentric interpretations this seems clear: Do not depict dinosaurs in any way except with complete scientific accuracy. Now, I don't actually disagree with this as for paleoart to be of value it must be done properly. If you're illustration for a paper, journal or media project then you've got to get your hands dirty and do the research and make sure what you're producing is scientifically accurate - a scientific illustration. But . . .

. . . art is not science. It's a creative as well as technical process and regardless of how scientific you want to be all that information and data is going to passed through that wonderful, complex and mysterious organ that is the human brain. When an artist creates a work they're bringing a whole panoply of conscious and subconscious influences to what they put down on paper, model in a computer etc. In the case of technical drawings for papers etc this filter is dialled right back (it has to be - no room for flights of fancy here) but for much of the rest of the time it's on and working flat out, bringing all an artist's experience and philosophy to bear on the subject at hand. Even in paintings and illustrative reconstructions the filter will be hard at work and is in a very real sense creating a mini-mythos that enables the artist to imagine the creatures he's depicting as living, breathing animals. Even if the animal is depicted accurately there is going to be one whole shedload of speculation and supposition in that depiction, any of which could be false.

It's fair to argue that so much is unknown we should be daring in our art. The work of such artists as Luis Rey and Bill Stout could be called mythological as a large amount of the reconstruction is speculative, the execution is stylised yet these images give life to the animals and communicate the work done by palaeontologists in the labs and in the field. These accurate reconstructions but also highly speculative in many ways; they are also powerful examples of good paleoart.

So perhaps it's OK, even unavoidable to create a certain mythology when depicting ancient life as living plants and animals. Perhaps my picture does convey something of the savagery of a large carnivorous dinosaur in a feeding frenzy (or perhaps it fails utterly?), gives some insight into it's inner nature - a subject that will forever be dominated by speculation. I would suggest it would even be desirable in paleoart to have some real left-field artist representations, something more abstract that might communicate the more speculative aspects of ancient life via emotion, intuition and the other influences artist have always brought with them to their work. In some respects, any and all reconstructions could be called false, but perhaps within the field we call paleoart there is place for more than purely scientific interpretation. No art exists in a vacuum, paleoart sits in an uneasy position between science and art and by definition consists of both in terms of technique and philosophy.


So should we call it paleo art, or scientific illustration?

The original ink sketch.

*I have no firm idea what this is. Sort of the quintessence, the nature, the theropodian id. It is by it's very nature undefinable, but it must have been there as it's there in every other living thing, from protists to blue whales.

Kicking the opposition into touch

Mike Taylor and Matt Wedel of SV-POW!, along with co-author Richard Cifelli have just published a paper describing a new sauropod from Utah, Brontomerus mcintoshi. It's a superbly presented piece of work, with excellent photos and drawings of the fossil along with some excellent paleo-art by Francisco Gascó, showing an adult Brontomerus mcintoshi booting a marauding Utahraptor into the air in defence of it's accompanying juvenile.

As might be expected, the media have picked up on the possibility of a calcitrant dinosaur (I'm resisting the urge to make a football/Aston Villa joke here) and the BBC have an article on their web site entitled Dinosaur named 'thunder-thighs', a reference to the name as "bronto" means "thunder" and "merós" (both from the Greek) means thighs; excellent nomenclature gentlemen!

The BBC site also features an interview with Dr. Mike Taylor along with some animations of the artwork, which are 3D models composited on a photograph. There are several exponents of this style of paleo-art around at the moment, notably Julias Csotonyi, whom I mentioned in the post last week about the dead elephant documentary. Of course this particular method of depicting dinosaurs has been around for a while and Jane Burton created some memorable images in the book she illustrated (text by Dougal Dixon) The Age of the Dinosaurs: A Photographic Record . These images are studio set-ups with what I assume to be painted and sculpted cut-outs of the various subjects in them to give a quite natural look; I will be writing about this book in the future.

Modern technology means we can now create very natural looking scenes by combining computer models with real-world environments that have been filmed or photographed. This is a  technique that has become very familiar and mainstream ever since a certain brachiosaur strode onto our screens and changed everything, and it's great to see it used for a reconstruction in a paper.

3D dinosaurs? Hmmm, I must post about that at some point . . .

Refs:

Taylor M., Wedel, M., Cifelli, R. (2011) A new sauropod dinosaur from the Lower Cretaceous Cedar Mountain Formation, Utah, USA. Acta Palaeontologica Polonica. 56 (1).