<blockquote>
<p>Should we make [deep-learning] tag a synonym for [neural-networks]?</p>
</blockquote>
<p>No. The neural-networks tag is for both SNN and DNN, whether NN tagged questions should be retagged deep or shallow is a different question. So keep NN when the question applies to both and use deep or shallow when the other isn't applicable, same for the resolution (width).</p>
<p>See:  <a href="https://stats.stackexchange.com/q/222883/197829">Why are neural networks becoming deeper, but not wider?</a></p>
<p>Depth and width, or lack thereof, are differentiating factors.</p>
<p><a href="https://stats.stackexchange.com/q/214360/197829">What are the effects of depth and width in deep neural networks?</a></p>
<p>See this question from 3 1/2 years ago: <a href="https://stats.stackexchange.com/questions/182734/what-is-the-difference-between-a-neural-network-and-a-deep-neural-network-and-w">What is the difference between a neural network and a deep neural network, and why do the deep ones work better?</a> (100 up / 0 down) and the useful image included in <a href="https://stats.stackexchange.com/users/28666/amoeba">amoeba</a>'s (Jan 18 2018) <a href="https://stats.stackexchange.com/a/184921/197829">answer</a> with 136 upvotes, 0 down, (and the other 9 answers barely scoring 2 dozen votes total):</p>
<p><a href="https://i.stack.imgur.com/trj4L.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/trj4L.png" alt="Amoeba's image #1" /></a></p>
<blockquote>
<p>... for real-world tasks deep architecture are often beneficial and shallow architecture would be inefficient and require a lot more neurons for the same performance.</p>
<p>But it's far from proven. Consider e.g. Zagoruyko and Komodakis, 2016, Wide Residual Networks. Residual networks with 150+ layers appeared in 2015 and won various image recognition contests. This was a big success and looked like a compelling argument in favour of deepness; here is one figure from a presentation ...</p>
</blockquote>
<p><a href="https://i.stack.imgur.com/iVURh.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/iVURh.png" alt="Amoeba's image #2" /></a></p>
<p>Article by Bernard Marr &amp; Co.: &quot;<a href="https://bernardmarr.com/default.asp?contentID=1789" rel="nofollow noreferrer">Deep Learning Vs Neural Networks - What’s The Difference?</a>&quot;. While I disagree that the <em>line</em> should be <strong>3</strong> I agree that there's a difference.</p>
<p>Quora Q&amp;A: &quot;<a href="https://www.quora.com/What-is-the-difference-between-Neural-Networks-and-Deep-Learning" rel="nofollow noreferrer">What is the difference between Neural Networks and Deep Learning?</a>&quot;:</p>
<blockquote>
<p>Nicolas Neubauer, PhD in network mining &amp; analysis<br />
Answered May 27, 2015<br />
&quot;Neural networks&quot; can be used to refer to the whole class of machine learning architectures where individual units are connected via weights and those weights are adjusted as the network is trained. In that sense, deep learning is just a particular branch of network architecture and training.</p>
<p>In a more narrow sense, neural networks might refer to the &quot;old-school&quot; way of constructing and training networks, where you have few layers (typically input, output, and 1 or 2 layers in-between), and then deep learning is the &quot;new&quot; way of doing this.</p>
</blockquote>
<p>Wikipedia: &quot;<a href="https://en.wikipedia.org/wiki/Deep_learning#Overview" rel="nofollow noreferrer">Deep Learning</a>&quot;:</p>
<blockquote>
<p><strong>Overview</strong></p>
<p>Most modern deep learning models are based on an artificial neural networks, specifically, Convolutional Neural Networks (CNN)s, although they can also include propositional formulas or latent variables organized layer-wise in deep generative models such as the nodes in deep belief networks and deep Boltzmann machines.</p>
<p>In deep learning, each level learns to transform its input data into a slightly more abstract and composite representation. In an image recognition application, the raw input may be a matrix of pixels; the first representational layer may abstract the pixels and encode edges; the second layer may compose and encode arrangements of edges; the third layer may encode a nose and eyes; and the fourth layer may recognize that the image contains a face. Importantly, a deep learning process can learn which features to optimally place in which level on its own. (Of course, this does not completely obviate the need for hand-tuning; for example, varying numbers of layers and layer sizes can provide different degrees of abstraction.)</p>
<p>The &quot;deep&quot; in &quot;deep learning&quot; refers to the number of layers through which the data is transformed. More precisely, deep learning systems have a substantial credit assignment path (CAP) depth. The CAP is the chain of transformations from input to output. CAPs describe potentially causal connections between input and output. For a feedforward neural network, the depth of the CAPs is that of the network and is the number of hidden layers plus one (as the output layer is also parameterized). For recurrent neural networks, in which a signal may propagate through a layer more than once, the CAP depth is potentially unlimited. No universally agreed upon threshold of depth divides shallow learning from deep learning, but most researchers agree that deep learning involves CAP depth &gt; 2. CAP of depth 2 has been shown to be a universal approximator in the sense that it can emulate any function.[citation needed] Beyond that more layers do not add to the function approximator ability of the network. Deep models (CAP &gt; 2) are able to extract better features than shallow models and hence, extra layers help in learning features.</p>
</blockquote>
<p>Etc.</p>
