Hi ,
When you're fitting a machine learning model, one of the most important things to consider is goodness of fit.
If the model is overfit, it will describe the noise in the training data, rather than the underlying relationships, and fail to generalise to unseen situations.
If it's underfit, it won't be
complex enough to capture any patterns at all, and will also perform poorly.
Like with Goldilocks, the best solution is typically somewhere in the middle.
Here's the thing...
As human beings, we are constantly creating mental models based on the data available to us - without even realising it. And with those models, we also need to be cognisant of goodness of fit.
Making sweeping generalisations about a person or place based on a few limited interactions is rarely useful - underfitting to your data by constructing models of the form: "All <nouns> are <adjective>".
Yet, trying to generalise from trivial details – and overfitting to your data –
will yield equally poor results. Eating a donut each day or only ever wearing black will not make you a tech billionaire, regardless of whether Steve Jobs or Elon Musk did or do the same.
When fitting a machine learning model, it can be difficult to tell if the model is overfit or underfit without running tests. That's why best practice is
to validate model performance against an unseen test dataset before deploying it into production.
Taking time to validate our own mental models should also be seen as best practice, if we want our mental models to be of any practical use.
Talk again soon,
Dr Genevieve Hayes.
