Invisible Visual Effects – invisible or impossible?

​

Visual Effects, also known as VFX, is a more complicated process than you might think. It is primarily about manipulating or creating photos outside the context of the live action in the movies. In recent years, the popularity of visual effects has grown significantly, so now it is used in films, computer and video games. I will look at the techniques and approaches of producers and visual artists to achieve invisible visual effects. Is it a dream or maybe what a few years ago seemed unreal to achieve, becomes history?

Without the visual effects, the film's story and its appeal would not be as absorbing and exciting nowadays. It is necessary to integrate the live footage with the generated photos (i.e. combining special effects with digital effects). If the connection is made as skilfully as possible, then an environment with a very realistic appearance is created. With the help of visual effects, created environment, the implementation of which cannot be made in reality, among others due to high costs and risk during its creation and use, huge time-consumption or lack of practicality in a later application. Visual effects created entirely based on computer technologies.

When it comes to VFX, these can be divided primarily into special effects and digital effects. Special effects, it is a group that includes all visual effects created in live action. When it comes to digital effects, it is a bit more complicated. In this case, various photo creation or manipulation processes, for which the photographic resources are made are fundamental. In the case of digital effects, the integration of still photos and computer-generated images is often performed. This allows you to create very realistic portrayals that were impossible to develop in the real world for unlawful reasons. The digital effects include matte images and frames, which can be both digital and traditional, used primarily to create backgrounds for 3D characters. Animation, not assigning movements to any objects and signs using 2D or 3D technology, composing, i.e. combining various kinds of visual elements from very different sources. Composing helps create the impression that all the features that make up an effect are part of the same scene. Modelling is the entire process of creating models in 3D technology. It is mainly about creating props or characters that are not suitable for real creation, which is an essential element of each scene. Modern software is used for this purpose.

Now let us go back to where it all starts. In 1986 Tom Gunning introduced us to the term "cinema of attractions". In his article, he mentioned, "the cinema of attractions solicits a highly conscious awareness of the film image engaging the viewer's curiosity". Instead of focusing on the narrative, the range of "cinematic manipulations" involving trick effects, drew the most significant interest and curiosity. Gunning cautioned that we should not merely create an opposition between "attractions" and narrative but remember that "effects are tamed attractions". 

For the first time, special effects presented in the movies "The Enchanted Drawing" (1900) and "The Journey to the Moon" (1902). Models of stars and planets were made of clay or were puppets. Although the history of the progress of VFX goes back to XIX century, I want to focus on nowadays Visual Effects and what makes them invisible for the human eye. The best way to see technological progress is to compare the same films re-made in several years apart. You can see the gulf between the beginnings of cinema and today's productions. One of the best and well-known examples is "Dawn of the Planet of the Apes" 1968, re-made in 2014. Pluralsight, in its article, mention "The world of effects in films has come a long way from special effects to the dominated realm of visual effects." what is noticeable in "Dawn of the Planet of the Apes". From human in costume to "extremely realistic apes" which are CGI. The key to realism is "motion capture and the visual aesthetic like the rendering of the fur". The detail of texturing, modelling, muscle and fur simulation were critical as many of the shots were close-ups. Motion capture played a pivotal role to add final realism. At the same time, all these technologies go unnoticed. It allows the audience to experience the digital characters to deliver emotional performances.

Forrest Gump, 1994 is one of the best examples of visual effects utterly invisible for the audience in the late 20th century. Losing legs by Lieutenant Dan Taylor during the Vietnam War was shocking and raised many questions. To explain all the invisible effects and "celebrates the technical achievements" Kim Aubry (producer) in 2009 released "Getting Past Impossible: Forrest Gump and the Visual Effects Revolution". This 27-minute documentary presents how the visual artist approached the "tricks". It explained how they "cut" Lieutenant Dan Taylor legs.

The blue socks halfway up his legs helped the compositors edit out the unwanted part of the legs. When Lieutenant does 180 degrees turn on the ship's sidewall, compositors replaced the wall's hole with the CGI wood. To add even more realism and illusion that he genuinely has missing legs, visual artist model pieces of furniture, i.e. wooden cabinet, when he reached for the alcohol and coffee table, while he fell on the ground from the wheelchair. They marked the spot on the floor with a blue circle. Many takes were very complicated. Compositors removed actor legs by deleting or painting his legs and replacing this part of the frame with a "blank plate". The footage shot from the same vantage point, with the same camera and without actors on the set. The compositors removed or covered Lieutenant lower legs with a sample of a blank plate record. Integrating this blank plate sample is laborious, requiring a working frame by frame. However, hard and monotonous work results in an excellent outcome. It is unbelievable how visual effects can trick our minds.

Another great example to support my thesis in the movie "Harry Potter and Deathly Hallows: Part 2", 2011. "Were there any effects in this film?" writes Ian Failes in his article. When we ask ourselves this sentence after the movie, that means the world to visual artists. The hard work paid off, and they achieved the primary goal. Failes mention "275 shots ranging from such magical effects as CG apparitions, to set replacements and wand battles", this is an incredible number of shots in a relatively short period, "the shooting schedule was set at 250 days, the filming took 478 days to complete" (wikipedia.org). The author of the article focuses on "crucial" and one of the most critical shots in the film when Harry announced to Ron and Hermione that Voldemort knows they are looking for Horcruxes. The camera pans around the actors, while they are stepping out of the lake and change clothes. Matthew Twyford, the film's work supervisor, explains "it was originally just one long shot". After a closer look at the original scene, the only real elements are the lake (pond of water), rocks, grass field and actors. Ian Failes reminds us that the scene was cut up into five shots, each shot contains 954 frames. To obtain complex effects for the background, Baseblack Studio used the mountains and lakes gathered by Scotland production. The artists' team combined the sequences into panoramas reworked in Nuke to CG geometry to show the shots parallax. The Baseblack Studio compositors also had to contend with the dark green screen surrounding the scene, the green grassy mound, and actors’ clothes contained green elements. "It was a very complicated tracking task with difficult keys" notes Baseblack visual effects producer Kate Phillips. In this scene, the green screen was a key element to achieve a successful result. It helped compositors to smoothy combine everything in Nuke. "It stands to reason that Deathly Hallows Part 2 is the best-looking movie in the series. The budget was enormous, and it embodies a full decade's worth of advancements in visual effects and CGI technology." comment Nathan Sharp.

Next example of excellent invisible for the eye effects is Robert de Niro's de-ageing in the movie "The Irishman", 2019. "How exactly do you make old men young again?" one of the most controversial questions asked by Hollywood production. Good software, experienced team of visual artists and money seems to be the most straightforward answer. Let us look at where it all started. Netflix, ILM and Martin Scorses doubted that the well-known, up to these days, costumes, face markers and facial makeup would do the trick for The Irishman.

Martin Scorses (director) did not want to distract the actors with any face stickers or marks. Pablo Helman (IML VFX supervisor) said: "we will develop the technology." Netflix wanted proof they can do it before they entrusted them with 140 million dollars. Leonardo Estebecorena (associate VFX supervisor) from ILM mention the techniques were the next level of performance capture, the lightning acquisition, and the set scanning. More accuracy was needed than ever before. To sign the contract with Netflix, they did a test, filming an actor in New York and applying the techniques they came up. It worked! They split into two groups; the first group focused on the camera and captured, and the second group worked only with software. The camera set up is a fascinating technique. With the assistance of two more cameras, the primary camera creates the pictures with no shadows. Also, each of the cameras gives a different perspective of the specific performance which means better 3D translation. In the interview, they mentioned that visual artists goal was not to recreate the younger version of the actors but the younger version of the characters. In the first stage of de-ageing, the graphic artist used Medusa technology developed by Disney Research/Studios (Facial Capture System). The actor had to sit on the chair and go through a series of facial expressions. The software generated a series of models that could move from one facial expression to the other; simultaneously would solve the way that animation blend on a particular frame. Sonia Contreras (facial capture leader, ILM) explains how Flux works, the second most crucial software. The software's name stands "F" for a facial and "lux" for a lightning component.

Flux catches the actors' facial performance and take the image to create a 3D model of the shot from the set. Then visual artists can focus on biological aspects, i.e. pores, wrinkles, blood flow, facial muscles and the most critical De Niro classic mole. Flux team played the key role to get the best results, and they made it. The first public reaction was "so what am I looking at?" when you hear that question as a visual artist; you know you did everything possible to achieve that invisible effect. Overall, Helman's team (500 VFX artists) produced 1750 visual effects shots (almost 250 minutes of the total movie length). We can all agree on the techniques and software they used; it is the future of VFX: no more 138 face markers nor makeup. Actors can play their roles without any facial distractions, and the audience can enjoy the "magic" on the screen.

Before I come to conclusions, I think it is worth to take a quick look at the software evolution over the year. Without any doubts, software progression is a crucial element in the visual effects industry.

Autodesk Maya is usually the main reason behind the beautiful visuals in some astonishing films like Harry Potter and Dawn of the Planet of the Apes. The initial release dated in February 1998, over 22 years ago. Maya comprises of tools to help with animation, 3D modelling, movement designs and rendering. From the range of Autodesk products, 3DS Max is the next most used 3D display software. Autodesk 3DS Max is vital for television, movies, engineering, and in the world of computer games. Jumping to the postproduction, Nuke is a node-based digital compositing and visual effects application used for television and film developed by Digital Domain. From my own experience, there is no such thing I would not be able to do in Nuke. This software makes visual artists life easier; the only limitation is imagination. Although Nuke is not easy software, once learned do not stop compositors from achieving the most remarkable results possible. There is more VFX software on the market, but I think those three are the most outstanding and commonly used by visual artists. Developers are still working on improvements and refinements. It is not like they released this program a few years ago and frozen it in time. Autodesk releases a significant update every year, as does Nuke.

In conclusion, all the films I have mentioned, from Forrest Gump to The Irishman, show the evolution of invisible visual effects on an extremely high level. I believe that each example I presented together with the techniques and software used is an undeniable proof that the combination of all these elements with a team of excellent visual effects artists on top of that, is indisputable evidence that it is possible to achieve something invisible to the public. However, only in combination with perfect for the shots software and human work is this possible. It is also essential to be creative and correctly set the camera in the scene together with green screens, lighting, and excellent actors’ performance. Although it is usually a couple of seconds of the visual effects per group of artists, it means months of work for the best outcome. I hope I proved in this essay that invisible does not mean impossible.

RESEARCH/BIBLIOGRAPHY:

1. Cinema of Attractions, Tom Gunning, 1986

2. The Evolution of VFX in Movies: The 60s Till Now, Pluralsight, 29 January 2015

3. Incredible, invisible effects, IAN FAILES, 30 SEPTEMBER 2011

4. Wikipedia, https://en.wikipedia.org/wiki/Harry_Potter_and_the_Deathly_Hallows_%E2%80%93_Part_2

5. Harry Potter: 5 Visual Effects That Have Dated (& 5 That Still Look Magical), Nathan Sharp, 16 November 2020

6. DAWN OF THE PLANET OF THE APES | VFX Breakdown by Weta Digital (2014), 28 December 2020, https://www.youtube.com/watch?v=15lj1UTE7PA

7. Getting Past Impossible: Forrest Gump and the Visual Effects Revolution, Kim Aubry ,25 November 2009

8. How The Irishman’s Groundbreaking VFX Took Anti-Aging To the Next Level | Netflix, 4 January 2020, https://www.youtube.com/watch?v=OF-lElIlZM0

9. "We Were All Very Nervous": How The Irishman’s Special Effects Team Got the Job Done, Chris Lee, 9 January 2020

10. Medusa Facial Capture System, https://studios.disneyresearch.com/medusa/

11. Autodesk Maya, https://www.autodesk.co.uk/products/maya/features?plc=MAYA&term=1-YEAR&support=ADVANCED&quantity=1

12. Autodesk Maya, Awards, https://en.wikipedia.org/wiki/Autodesk_Maya

13. These VFX software are used frequently to create out-of-the-world visual effects, 13 March 2020, https://www.maacindia.com/blog/these-vfx-software-are-used-frequently-to-create-out-of-the-world-visual-effects

14. Nuke,
https://en.wikipedia.org/wiki/Nuke