Limitations & ethics

An honest account of what this model cannot do.

A recommender is a product decision as much as a model. These are the limits and trade-offs a reviewer should know before trusting it, stated plainly, because hiding them is how recommenders cause harm.

Modelling limitations

Long-tail items are near-unrecommendable. Collaborative filtering has almost nothing to learn for artists with very few listeners. This caps achievable recall and is a property of the data, not a bug. Reaching those artists requires content features (audio, tags), which this pure-CF system does not use.

Popularity bias and feedback loops. Like all collaborative filtering, the model leans toward already-popular artists. If its own outputs fed back into training, that bias would compound. It is not mitigated here (offline, single snapshot), and it is the main equity concern: niche tastes and small artists are under-served.

No temporal signal. The snapshot has no usable timestamps, so the model cannot capture trend, recency, or a listener’s taste drift over time.

Stale data. Last.fm-360K is a research snapshot of past listening, not representative of today’s catalogue or of current behaviour. It is a methodology demonstrator, not a production model.

Coverage vs accuracy is a product call, not a model fact. The served configuration favours accuracy over catalogue coverage. Rather than hard-coding a single answer, the system exposes a diversity lever (MMR re-ranking) so a product owner can choose where to sit on the accuracy-vs-discovery frontier.

Fairness and ethics

  • No demographic attributes are used or available, so the model cannot directly discriminate on protected classes. The salient equity issue is the indirect popularity bias above.
  • Artist exposure. Concentration this steep (Gini ≈ 0.67 in the data) means a recommender can either reinforce winner-take-all dynamics or deliberately widen exposure. That is a values choice, surfaced by the coverage/novelty metrics rather than buried.
  • Scope. This is an offline demonstration on a research dataset. It is explicitly out of scope for real-time serving, high-stakes decisions, or any use where a wrong recommendation carries real cost.

Reproducibility

The full pipeline is deterministic given seeds (BLAS pinned to one thread) and rebuilds end to end:

python -m src.data_360k        # build the Last.fm-360K core
python -m src.harness.make_split
python -m src.harness.run_session
python -m src.harness.confirm_holdout   # one-time, human-gated
pytest                          # 137 tests, incl. toy-validated metrics

Notebooks 00-08 reproduce the EDA, the harness validation, the disciplined search, and the 2k → 360K scale-up. Every figure in this report is generated by that pipeline.

See the machine-readable model card for the same limitations in a structured format, and the live app to interrogate the model directly.