Where Machine Learning Projects Go Wrong in London

The recurring mistakes that keep a model in a notebook, and what a working system looks like instead.
Where Machine Learning Projects Go Wrong in London

A model scores well on a tidy sample, the team celebrates, and nothing about the business changes. That is the most common ending for machine learning work in London, and it rarely fails at the modelling stage. It fails earlier, in decisions nobody made explicitly.

Treating it as a modelling problem

The default instinct is to start with the algorithm. Pick a framework, tune some hyperparameters, chase a better score. But machine learning is a data problem long before it is a modelling problem, and the questions that decide the outcome are dull ones: are there enough labelled examples, does the target variable mean what everyone assumes, is there leakage between training and test sets, does the sample reflect the population the model will meet in production. Teams tend to discover the answers in month three, once the budget is committed, rather than in week one when finding out is cheap. A model built on data that cannot support it is not a model that needs improving. It is a project that needed a different question.

Scoping the notebook and not the system

A notebook proves a concept. It does not run on a schedule, it does not serve predictions to a product, and it has no owner when it breaks. The gap between a promising notebook and a deployed system is the data pipeline, the serving layer, the versioning, the rollback plan and the monitoring, and that gap is where most funded work quietly dies. The tell is simple: if nobody scoped those pieces at the start, they will not appear later out of enthusiasm. The notebook ends up in a repository, and the repeated decision it was meant to improve is still made by rules or by instinct.

Assuming a deployed model stays good

Fraud patterns shift, user behaviour moves, catalogues change. A model trained on last year’s data is slowly answering questions about a world that no longer exists. Without drift detection, that decay is invisible, and the first person to notice is usually a customer rather than a dashboard. This is not an edge case reserved for large systems. Any model making a repeated decision in a live product is exposed to it, and the cost compounds quietly month over month.

Confusing machine learning with the wider label

AI describes systems that plan, reason or handle language. Machine learning is the narrower engineering discipline where a model learns patterns from data instead of following hand-written rules. The practical test: if the system improves its predictions as it sees more data, it is using machine learning. Getting this distinction wrong leads teams to expect a rules engine to learn, or to buy a platform when the actual need was a trained model on their own history. It also muddies what to ask a supplier, because the two kinds of work have different failure modes and different costs.

What good looks like

A sound result is unglamorous. The data was checked before anyone trained anything. The model is measured against a baseline the business understands, on a holdout set that reflects reality. It runs on a schedule, serves predictions with latency targets and rollback, and its performance is tracked against live outcomes. There is documentation covering what it assumes and how it fails, written so the team can own it afterwards. Whether the work is a recommendation system, a predictive model or anomaly detection, that shape holds. For teams building retrieval or support tooling on top of their own data, the same discipline applies to RAG development.

If the pattern above looks familiar, the full guide sets out the feasibility assessment, the pipeline stages, the evaluation approach and the monitoring that turn a dataset into a deployed model — read it at machine learning development in London.


Write a comment