I too have been struggling a lot with this question and therefore I went through many blogs/tutorials and saw that there are many ways to do it.
But then the second question which hit me was that my model was in smooth preview mode (No 3 in maya) so in that case how to render. As coz smooth preview mode works only with mental ray render and mental ray will NOT render Wireframe. Wireframe can only be achieved via Maya Vector.
So after going through lots of tutorials and experimenting I came up with this method. I am writing this tutorial for myself so that it can be of use to me in future (Maya is a complicating thing to remember) :P
First of all. Its a good idea to save ur file in a differnt name and call it wireframe as the model have to be altered (So probably not the best idea to do it in your original file)
Second, Before you proceed make sure to light your scene as much as possible.. Assign ONE differnt LAMBERT material to all your objects in the scene and you are free to delete all the textures and shaders nodes as we longer require them. Your scene is going to be crazy heavy so best to "Optimize your scene" before proceeding. It will do your task of making the scene as light as possible.
Now straight to Instruction:
1> If u have any camera to set, Set your camera and lock it. (Imp)
2> If u have not, Assign ONE new material (LAMBERT), call it wireframe and color it full white [All the objects in your scene will have only that material assigned to it]
3> Now confirm face and Normals of your Model.
4> Save your file (Save as)
5> Select your model(all of it) and combine it [the objective is to make sure your scene has only one object]
6> Optimize your scene (This is the last time you are doing it, No more optimization after that or deleting history)
7> Now goto "EDGE" Mode and select all the edges in your scene.
8> Now, with the edge selected, go to CREATE>QUICK SELECTION SET> Give a name to it and Click OK (save your Scene)
9> Next, Make your model in the normal selection mode again and click MESH>SMOOTH (and click on the small box beside that)
10> Apply Division 1 or 2 (according to your scene performance) [Do not delete History]
11> Now select your model and apply NORMALS>SOFTEN EDGE
12> Now you have to seelct your original edge. So goto EDIT>QUICK SELECTION SET> Your set name
13> Now go to NORMALS>HARDEN EDGE [So that ony the hard edge gets rendered out]
14> Make sure Vector Render Plugin is loaded. [Go to WINDOWS>SETTINGS/PREFERENCE>PLUG-IN-MANAGER>VectorRenderer.mll]
15> Now open Render Settings window and choose "Maya Vector" & under Maya Vector tab, go to EDGE OPTIONS - Click on "Include Edge"
16> Give EDGE Style to "OUTLINE"
17> Give Weight to 0.05 to 1.00 depending upon your preference
18> Give "Edge Color" to Black
19> Make sure "Back Face Culling" is checked ON (Not necessarily. Please experiment with it) Leave evrything as Default
20> Now RENDER!!!
Incase of any question do mail me or leave comment. Hope this was helpful.