<p>Brian Borchers provided <a href="https://stats.stackexchange.com/a/23132/35989">very nice and detailed answer</a> comparing gradient descent to other optimization methods in machine learning describing when and why it is used.</p>

<p>[Bounty awarded 23 May 2016]</p>
