<p>Thanks to Sycorax, we have an excellent question which can be used to close as duplicate (most of) those questions about neural networks which can't overfit the training set:</p>

<p><a href="https://stats.stackexchange.com/questions/352036/what-should-i-do-when-my-neural-network-doesnt-learn">What should I do when my neural network doesn&#39;t learn?</a></p>

<p>This helped to close a lot of "my network doesn't work, please fix" questions:</p>

<p><a href="https://stats.meta.stackexchange.com/questions/5273/whats-the-best-way-to-answer-my-neural-network-doesnt-work-please-fix-quest">What&#39;s the best way to answer &quot;my neural network doesn&#39;t work, please fix&quot; questions?</a></p>

<p>I think we would also need a similar question for cases when the validation loss doesn't decrease (or does so only slightly). I tried to look for one, but I couldn't find it. Do we have one?</p>

<p>If the answer is "No", I'll write one with the same aim as Sycorax's. Now, to abuse the usual paraphrase from Tolstoy, "training issues are all alike; every generalization issue is particular in its own way", so it might be harder to give good answers to such a question, but I think that, if framed the right way, it could work. For example:</p>

<p>"My neural network doesn't reach the generalization error it should be able to reach, based on reputable sources or experiments with very similar NNs: which things should I check"?</p>

<p>For example, the first thing you need to check in this case is <strong>if</strong> you're overfitting the training set, and <strong>when</strong> (at which epoch) the training and validation losses start to diverge. There are a few questions where the OP only reports the accuracy on the validation set at a fixed number of epochs, which is pretty useless.</p>

<p><strong>Update:</strong> this question has been created: <a href="https://stats.stackexchange.com/questions/365778/what-should-i-do-when-my-neural-network-doesnt-generalize-well">What should I do when my neural network doesn&#39;t generalize well?</a></p>
