This tutorial covers how to render an ocean scene using nothing but a polygon plane and an HDR image connected to a Skydome light. The detail for the ocean shader will come from a vector displacement map. There are a variety of attributes to set when using displacement mapping. However, it is very simple to set up and get convincing results using the right settings.
The vector displacement map used is a 32-bit EXR image. This means it is able to retain the highest quality for the vector displacement effect. To get a more realistic wave effect, we will use a vector displacement map to displace the sea waves. This will displace the sea geometry around a direction different from the normal, unlike a traditional displacement map which displaces the geometry in one direction only.
The ocean shader is not fully transparent. This means that non-glancing rays will not reflect and will instead refract and go through the water (like in real life), but since it's not transparent, we instead terminate the rays and get black.
Shading normals can cause problems when the shading normal points away from the ray even though the geometric normal points towards the ray. In this case, the shading (Auto Bump) normal says that we are hitting the backside of the water and should reflect down, while of course in actuality the ray is hitting the top of the water mesh.
At grazing angles real water is nearly 100% reflective, but even so, it may take several bounces for the light to get out of the valley behind a wave crest, and even after several bounces it may reflect something that is higher up from the horizon than what the surrounding water reflects.
A more realistic look is to make the water transparent and then place a plane below the water that has the color of ocean water. In the example images below the ocean floor's Standard shader has an Emission Scale of 1 and a blue/green Emission Color. Increasing the Subdivision Iterations has also helped to reduce the number of black patches on the displaced sea waves.
The dark patches behind the wave crests (self-reflection) can also be improved by increasing the Specular Ray Depth in the Render Settings.
Light that penetrates the ocean surface is 'refracted' due to the fact that light travels faster in the air than in water. When the light reaches the water it will be scattered or absorbed by particles in the ocean. As the ocean becomes deeper (denser), the water will decrease the depth of light penetration. We can replicate this effect quite easily using the Transmission attribute in the Standard Surface shader. The effect can be seen when placing an object inside the ocean surface. We can see that the longer light travels inside the mesh, the more it is affected by the blue Transmission Color.
When applied to the ocean shader we can see the effect that Transmission Color and Depth has on the white cylinder inside the closed surface ocean below:
That concludes this tutorial on how to create a believable ocean scene. Using the correct displacement settings in conjunction with a high quality 32-bit vector displacement and the correct Standard shader settings should give you something that looks realistic. Why not try adding depth of field or change the camera lens to something like a fish eye in order to add more visual interest to the scene.
Below you can see how much more detail can be achieved by using a vector displacement map to create the wave effects compared to using regular geometry:
An .ass file is available here.