Roto Paint – Nira's VFX Blog

In this lesson, we saw the different techniques that can be used for 3D project, such as patch projection, coverage projection, or nested projection, and we also analysed how to add texture and lighting onto a 3D object as well as the general problems we can encounter with this.

In 3D tracking, we need to try to avoid to include the sky, as it would give us problems later on, in the same way that we avoid objects that move or reflections in roto.

When adding a ‘rotopaint’ to a card in a 3D space, we need to first freeze the frame with a ‘frame hold’ node at the best position in the sequence for visibility and tracking a specific point. Then we add the ‘rotopaint’ or the patch we need, and add another ‘frame hold’ to ‘unfreeze’ the frame. Then we premultiply it to create an alpha and use a ‘project 3D’ node to project it in our card (the ‘project 3D’ node must be connected to the projection camera and another ‘frame hold’ node). Lastly, we connect our card to the ‘scanline render’ node which will be merged with the main plate.

In order to add texture to a ‘card’ in 3D space, we will use the same method as before, but this time we will take the texture or picture that we want to add which we can ‘colour correct’ and ‘grade’ if needed, to then ‘roto’ the part we want to add from it, premultiply it, and with ‘corner pin 2D’ we will place it in the perspective we desire. Then we will ‘transform’ it to the dimensions we want and ‘merge’ it to the main plate after adding a ‘frame hold’. Lastly, we need to ‘copy’ the roto and premultiply it so we can project the alpha to our ‘card’.

If we want to roto something in the scene to change its features (colour correct, grade, etc), we can do the same as we did with the ‘rotopaint’ but in this case we adjust the roto every 10 or 20 frames. We do not need to adjust the roto every frame as it will follow our match move previously done so just a few adjustments should be sufficient.

When we have several 3D projections that we want to put together, we can use ‘Merge mat’ node, as if we use a regular ‘merge’ node, the quality of the image can decrease and look different.

After seeing these 3D projection techniques, we were asked to practice them using the following a footage of a street provided by the lecturer. For example, we could add something on the wall or floor, change the windows texture, colour correct a specific element of the scene, etc. This is the result of my practice:

When 3D projecting on top of a 3D object or artefact, the types of projections we can use are:

Patch projection
Coverage projection
Nested projection (projection inside another projection)

We can find some issues when doing artefact projections that can be solved we the following techniques:

Stretching problem: texture is stretched and not showing in the correct place. This issue can be fixed adding a second camera projector on top.
Doubling problem: texture is doubled. We can fix it doing two separate projections.
Resolution problem: texture look pixelated. We can use ‘sharpen’ node to solve it, however, we can also use a more efficient solution which is adding ‘reformat’ node and set the ‘type’ as ‘scale’, to then link node to ‘scanline render’ which would be the connected to a second ‘reformat’ node with the resolution of the original plate.

Lastly, we also saw how to build a 3D model taking as a reference a 2D image. Using ‘model builder’ node, we can create and adjust cards following the perspective of the 2D image, to then ‘bake’ this geometry into a 3D space. We can add ‘point light’ nodes to set illumination with different intensity, colours, and cast shadows. Another illumination node is the ‘direct light’ which is used as a filling light directed to a specific point or direction.

Once we finished reviewing this week’s theory, we were also asked to make the roto of the hole in the scene of the Garage project and to remove the markers with patch projections. I made the roto pretty quick and had no issues with it, but I struggled with two specific markers clean up: in the two markers positioned by the hole in the wall, when I added the roto, the patch made with rotopaint was showing outside the roto boundaries (right on top of this roto), so it was showing the wrong patch.

After asking the professor for some help, he figured out that I missed the lens distortion node on both the beginning and the end of the clean up set up (to undistorted the scene and the redistort it back).

Final nodes graph set up with correction

Another issue I noticed is that the patches added on the floor marks were showing through the roto of the wall. I asked the professor again and found out that this part needs to be merged differently as it is outside the roto. So added a ‘merge (stencil)’ just to these part of the clean-up, then ‘shuffle (alpha-alpha)’ and connected it to the roto ‘scanline render’ node. This will create an stencil of the patches taking the roto as reference and it will not show through the wall.

Clean-up cards visible through roto error

Final clean-up + roto

I had a lot of troubles with this homework and spent a lot of time trying to figure out why it was not working, but I feel that this struggle was useful to familiarise a bit more and feel more confident towards the nodes system used in Nuke.

Week 9: Blur, Defocus, and 2D Clean-up in Nuke

In this session, we learnt how to use ‘Blur’ and ‘Defocus’ in a scene and how to do a 2D clean-up using ‘Roto Paint’, ‘Difference’, ‘Regrain’, and ‘Grain’ tools.

In order to add realism to a scene, it is a good technique to add some ‘Blur’ or ‘Defocus’ to it. However, depending the desired effect, we use one or the other. We use ‘defocus’ to emulate what happens with a real lens when unfocused, so since this is a more realistic and natural effect than ‘blur’, this is more commonly used for a more cinematic and more visible effect. On the other hand, ‘Blur’ is used when we need to defocus a colour or something minimum that is going to be barely visible (more for correcting purposes rather than effect wise).

‘Z Defocus’ and ‘Z Blur’ are used to defocus or blur specific areas of the plate and can be also used to blur or defocus taking in consideration the depth of the shot when the alpha is converted to depth. With these nodes we could also defocus or blur following a shape like a disc, bladed or following a roto we made. These nodes can be used together with ‘Convolve’ node in order to defocus or blur with a roto shape in different forms.

Nuke is also used for cleaning up a scene. This can be made using ‘Roto Paint’ node with which we can paint, clone, blur, dodge, and burn specific areas of he shot. After this, we could add a ‘Difference’ node to subtract the alpha taken from the ‘Roto Paint’ area followed by a ‘Copy’ and’Premult’ nodes. Also, we could add a ‘Frame Hold’ node to freeze the reference frame where we are going to do the roto painting.

Once we added the patch or correction to our shot, it is good practice to add a ‘Grain’ effect to match the grainy texture of the video and the patch blends in. We can use ‘Grain’ node which is applied through the alpha so it does not affect to whole plate but just the alpha area, or the ‘ReGrain’ node which will affect the whole plate as double grain (so it cannot be applied multiple times).

This week’s task was to practice what we learnt today trying to do a clean-up of the school shot provided: removing some papers that are on the wall, adding some roto paint in the side of the lockers, adding something to the background door (in my case I added some animated text), adding something in perspective in the left side doors (I added a video of what it looks like a circular magic portal), and adding something interesting on the floor (I added another magic portal).

To start, I wanted to remove some papers from the pin board on the right. To do so, I added a ‘Roto Paint’ node and used the ‘clone’ tool to paint on top of the papers using the texture of the board. Then with a regular ‘Roto’ node I created the alpha of the painted area followed by a ‘Filter Erode’ to soften the edges and a ‘Premult’ to transform it into an alpha. All of this has been done with a ‘Frame Hold’ node so it is easier to build up the roto. Then I tracked the area with 4 tracker points and created a ‘Transform (Match Move)’ tracker node to match the move of the scene. Finally, I added the ‘Grain’ node to match the grain of the Roto Paint with the scene grain and merged it with the main comp.

Secondly, I added an animated ‘Roto Paint’ to the side of the lockers. I used the already existing tracker node that was used to remove the poster that was in the same place that I wanted to add the new ‘Roto Paint’. I created a ‘Transform (Match Move)’ tracker node and attached it to the ‘Roto Paint’ node with the animation. To animate the painting, I played with the colour and opacity adding key frames in these features. Then, I linked this the the main comp.

Animated Roto Paint at the side of the lockers

Thirdly, I added some text in the back doors tracking the area first and then adding ‘Corner Pin 2D’ first baked to fix the frame and then another one to match move the scene movement. I also added an animation to the text key framing the colour section, and the merge it to the main comp.

Animated Text fixed with Corner Pin 2D nodes

For both magic portals I used the same technique that we used last week with the ‘Planar Tracker’ and creating a ‘Corner Pin 2D (Relative)’ to fix the image to the area selected. I reformatted both clips and corrected the saturation and grades. Then I merged them to the main comp using ‘screen’ option so the black background disappears and there is an transparency effect in the colours.

Recent Posts

Recent Comments

Archives

Categories