It's going to be tougher than realistic human speech, and look how long it's taking us to get that right, but I don't think there's any reason to believe we can't eventually get avatars that are both realistic and 'human'.

Computer graphics still has a long way to go in general. Even the best video game graphics don't look real enough for me, the shadows and lighting are always off one way or another... I suspect that will continue to be true so long as GPUs can't manage global raytracing and global lighting.

Isn't this something you had written about before? If not, then I had read about it before, maybe a year ago--that is what drew me to this article when I visited Slate this morning.

If you have written about it before, did you do a professional piece on it? What was it? If so, is it a tactic of journalists to revisit (rehash) ideas into new articles?

Does the Uncanny Valley apply only to visual images? I think our voice synthesis is starting to get there, and some of it is actually fairly good, but again it sounds "dead". I think we may have overcome any uncanny aversions to artificial foods, scents...

hmmm, my last comment sounds like an attack, it's not, just a curious question.

Very interesting article. How does a character like "Max Headroom" fit in. While he was very human in appearance maybe the jerky, stutter style made me like him. Human in appearance but very mechanical in action.

Good article.

Maybe it's jarring when we are familiar with the individuals these dopplegangers are based on. Take the intro trailer for Onimusha 3. I don't have a clue who Takeshi Kaneshiro is and as a result he remains an uber cool game avatar. It doesn't hurt being culturally programmed to assume a natural disconnect in voice and expression after a lifetime of badly dubbed Kung Fu movies. A great deal of leeway is afforded here. Jean Reno on the otherhand? A grisly parody of the familiar actor.

It's only when the virtual and real meet. Solid Snake from Metal Gear Solid is a natural fit in his virtual world and exists, in my mind, seemlessly within it. Had he been created in the likeness of Kurt Russell I don't think I could resolve the two.

I suggest to look here: http://www.arclight.net/~pdb/glimpses/valley.html

I forgot to note, although in traditional art, the practice of idealization was the norm, towards the end of the 19th century there was some experimentation in ultra-realism that did end up looking eerie, particularly so in sculpture. The sculptor Max Klinger intentionally used this to create Symbolist scultpures. Just the addition of pupils to the eyes makes realistic sculpture look more creepy; he did this, plus ad multiple tones to the skin. Another sculptor who pushed ultra-realism, though its uncertain whether he realized the effect was creepy, was Jean-Leon Gerome, an academic. His sculptures have no intended symbolist content, but the degree of realism make them appear like actual people frozen in marble.

There is also a contemporary British artist who creates sculptures out of wax to resemble actual people with full realism. Except in many of his works he distorts the features somewhat to make them more cartoonish; and the existence of such a real looking person with cartoonish features is creepy in its own way.

This worthy topic should be studied from a gaming business angle too. Starting from the mid-90's, there has been much fewer new genres of games coming from developers. As the industry matures, it grows more like the movie industry, where the studio/publisher wants more sure-win title to counter the inherent financial risk of making a game. Creative and financial resources shift to the most observable improvement: graphics. Undeniably, the market in general demands better eye-candy, but when that single factor dominates the whole industry, it becomes an indication that maybe soon the market will be fed up. Afterall, isn't that the reason we laugh at the later Rockies and First Bloods -- the audience can only take so much of Rambo killing another foreign soldiers or Rocky taking another punch? So, what if the CG character in Alias looks more Garner than the real Garner?

Another point I would like to make is that we have to remember when we say "real", we usually mean the characters acting "real" in the cut-scenes, which are thoroughly computer-animated. Another fight and action scene in which a player is able to control the movement of a character is stil very far away from anything being life-like, or as life-like as the library of motion capture allows it.

GL

Thanks for an interesting read.

I guess this might be a reason for the current resurgence of interest in 'retro' games .. after the initial novelty of 3D wore off, a lot of people realised they preferred the simplicity of the older, less realistic games; since they were less lifelike, the brain is perhaps more inclined to give them the benefit of the doubt and suspend disbelief, rather than being like "Look at that sprite walking, that's ridiculous, no one walks like that".

This is also perhaps another reason people didn't buy into a full CGI film like Final Fantasy, whereas, for instance, Gollum from the Lord of the Rings movies is a huge success: he's sufficiently far removed from a real human that, although a creepy character, doesn't have the shudder-factor of the almost-human.

Actually this macabre effect can work positively too. I remember, in the days when I would waste countless hours playing a certain popular 3D shooting game in head-to-head mode, the faces of the characters (including my friends' avatars that I was trying to kill) would have that same uncanny almost-human look about them. I think the loathing I felt towards these creatures made me play the game better.

These are all extremely cool comments! Excellent points about the history of art and abstraction vs. realism. Also, yes, there's definitely a difference between the quality of cutscenes and the quality of in-game animation -- though I personally think even most *cutscenes*, which benefit from hours and hours of patient construction, are still pretty ugly and un-human-looking -- when it comes to faces, at least.

Alfred, by the way, yes, I'd blogged about this before -- but never written about it journalistically!

Good article, good comments.

The naturalist portraiture of the Renaissance proves that illusionism is definitely possible, but it takes a lifetime of hard work to attain the craft skills necessary to pull it off consistently. Games are still a ways from achieving that because:

A) We need more good artists. A lot of visual artists in the game industry are here simply because they couldn't get work in a CG house like ILM or Pixar, and don't really care much about the medium of games and about working with its unique strengths and weaknesses.

B) The aesthetic stupidity that has conditioned developers and consumers alike to get more excited about bump mapping and higher polygon counts than more expressive, compelling, or generally higher quality art content.

C) Illusionism is even harder in three dimensions. The Greeks of the Hellenic period and the Renaissance guys did amazing work, but the unpainted, uniformly textured nature of the marble gives them a remote feeling (sort of like black and white photography) that steers them clear of the Uncanny Valley (although some of the classical Greek statues were originally painted). Only a few modern day wax sculptors are able to make something that doesn't end up looking like a creepyass Real Doll (link is not safe for work!).

Interestingly, illusionism in painting was at its peak right up until the point when photography was invented. Suddenly there was a machine that was capable of more or less the same depictive accuracy as a grand master. After some resistance, painting began to redefine its role in the visual arts, and the results were Impressionism, Cubism, and Abstract Expressionism.

Games need to stop confusing fidelity (more pixels, more polys) with quality if they're to become more than technological curiosities and marketing tools for other media.

Jp,

I just want to clarify some of the arguments I made with some visual aids I found. Fully photographic depictions of people were never sought after in the Western tradition, there was always a push to idealize since the Renaissance, and during the ancient era. In fact, Rembrandt was dismissed as a realist, and was all but forgotten until his works were rediscovered in the 19th century. The aim was to come to a Platonic ideal, a generalized form where features were geometrically simplified and harmonized, plus one where the line and color was manipulated to support the theme of the art. Here is an illustration, showing the 19th century artist Bouguereau painting from a live nude:
http://www.xs4all.nl/~frqnc23/images/ph_4.jpg
It becomes obvious how much the form was altered and abstracted from the real person. This can be realized more simply: you could never mistake a traditional painting for a photograph, no matter how much inclined you are to say its like life.

In fact, the Realist movement which arose in the 19th century, exemplified by Courbet among others, had the goal of going against all of these academic conventions to show what things actually looked like in real life (in addition to abandoning the mythological subject matter). The academic methods were found to create a slick surface and unreal imagery which the Realists sought to work against.

Realists were as unwelcomed by the conservatives in the art world, as much as was later modern art. Their artworks were found ugly, lacking poetry or beauty in their depictions. Similar things were said about Impressionism, and later movements. But the avant garde pushed in this direction to rid art of what they found as sentimental and what was often clichéd by relying on classical or religious themes. They believed using literary motifs in art caused these problems and eventually wanted art to focus on pure form, like music. Photography had a very limited role in the development of modern art, although it by no doubt was used as an argument. But, after all, figurative art started to become reappreciated in the 1970s to a position where its respected again today, and not thought of as mere human-made photography.

I also wanted to show some examples of art at the end of the century which pushed towards ultra-realism. A very fleshy sculpture by Gerome:
http://www.artrenewal.org/images/artists/g/Gerome_Jean-Leon/small/museum/The_Ballplayer.jpg
And a sculpture by Max Klinger:
http://www.pinakothek.de/images/13173_1187-l.jpg

I do agree though that part of the reason that video games have problems like this is because few people who work on them have a real sense of aesthetic. A lot of the 3d graphics just looks ugly

Hmm, very good points Brian. I'd like to make a distinction, and please correct me if this is off-base as you are clearly very well-informed, between realism as a method of rendering and realism as an approach to artistic depiction. As far as technical quality of the depiction goes, the Renaissance masters aren't too different from the 19th century Realists. The perspective is perfectly observed, the treatment of light and texture are highly competent, and details are rendered more or less (depending on the artist) to an extent that the image is what you might call "believable" (and yeah Rembrandt was ahead of his time in the attention he devoted to those details). Where they differ is in how much they idealize their subject matter. Realism in art was/is about stripping away the falseness and pretention of classical art and showing the Actual rather than the Platonic Ideal.

Videogame artists generally *do* want to idealize (nobody in videogame land has love handles or zits or a weak chin), but they try to achieve that with 19th century Realist methods, concentrating on the pores and nose hairs. They understand and capture the minutiae of the subject but not the essence, and you need both to create a good likeness.

Hey Clive, are you a journalist or a writer? Jo and I can't decide.....

Ah, I use 'em interchangeably. "Writer" connotes a more lofty profession -- i.e. it's a wider term that includes everything from novelists to playwrights to poets to journalists. "Journalist" designates you specifically as a guy who cranks out stuff for newspapers and magazines or TV or whatever, and thus seems less like a *calling* and more like a *job*. Me, I couldn't care less. I usually use "journalist" because "writer" sounds kind of pretentious.

You wrote this whole article and the fail to look at this issue with a focus on what's to come.

"Mouths and eyes don't move in synch. It's as if all the characters have been shot up with some ungodly amount of Botox and are no longer able to make Earthlike expressions."

Have you seen the Half Life 2 G-Man videos?

http://www.fileshack.com/file.x?fid=3279

If you get the creepy feeling from watching this it's because you are supposed to feel that way about someone as freaky sounding as this. :) My ultimate issue is that you seem to be pushing for "No CG human can ever look real enough to fool the human eye/brain." I don't find that to be the case, and this sort of graphics development is not on a linear path, hence we should see better looking, better synched humans sooner. It also means that the next ten years will bring us the high definition displays and pure number crunching power to make you wonder if you are watching an episode of The Sopranos you missed, or a whole new series made with virtual actors. The newest Nvidia card, which isn't even testing at the top of the heap, has roughly the same real-time rendering quality as Toy Story, and still fits inside your home pc. Correct me if I'm wrong, but we've got a bit to go before Toy Story is considered photo-realistic, though at the time much of what was shown on screen seemed very real.

Most of the issues you raise are a function of the viewer knowing that none of the video was ever real, and that has nothing to do with the code. Kuma:War is taking baby steps in blurring the "when is it a game?" situation, putting players in real-world events (though they fail horribly by not having a better engine, graphics included). I'd guess the other major hurdle to bypassing the "reality check" will be user interface. When you have a 3-D world "painted" on to the back of your eye so you can't divert your attention, and you're finally commanding the character in the game purely through natural movements of your own body, another piece in the puzzle of making virtual humans real will be in place.

p.s. "Lara Croft is another good example. Even as her games became more graphically precise, the designers left Croft as a very stylized figure, the better to have players identify with her."

To borrow from an abused political motto: "It's the chest, stupid." Who's going to dial down the boob-o-meter (thereby making the charcter more "graphically precise") when most of the gaming population is only playing this tired platformer to see the new breast physics?

P.P.S. I totally agree that the large majority of game designers have taken gameplay and strapped it behind them into the child seat between the other two gaming children, "online play" and "massively multiplayer." I think there are at least a few titles every year that really are using high-end graphics to bring gameplay to the forefront. It's only going to get better as realistic physics engines and high poly count models are the norm. Perhaps you should get away from the Xbox as the developer community for that console seems to be the worst to date. (Though the phantom may give them a run for their money.) The good games for the next 12-18 months are going to be on the PC, and the graphics cards / processors that will lead to the game developments I'm focusing on are only hitting shelves this summer.

"The newest Nvidia card, which isn't even testing at the top of the heap, has roughly the same real-time rendering quality as Toy Story, and still fits inside your home pc."

And it's worthless without Pixar-quality animators and artists making content for it. Solving artistic problems with technology NEVER, EVER WORKS!