Expo West 2024: BATTLING THROUGH BAD DATA: HOW TO SUCCEED IN AI

Why AI Projects Fail

Dr Janet Bastman, Napier AI chief data scientist, opens with a rhetorical question about money wasted on doomed AI efforts. She cites Memorial Sloan Kettering’s IBM Watson cancer project—launched with fanfare, scrapped by 2018 after doctors called recommendations unsafe, costing about $62 million over six years as a cautionary example.

Rejecting Magic-Box Thinking

Bastman contrasts vendor hype—gather data, feed AI, get answers—with data science discipline. She argues science is magic that works, and that niche medical summarization remains hard even for modern LLMs. Failed projects often skip user feedback, doctor validation, and iterative checks.

Defining and Fixing Bad Data

Bad data means information unfit for purpose: inconsistent, unbalanced, inaccurate, or unrepresentative samples that mislead models. Bastman urges applying scientific principles—clear problem definition, representative sampling, and honest assessment of whether datasets support confident answers—before scaling investment.