Unlocking the Power of Big Data in New Product Development

One of the most promising applications of big data is feeding new product development (NPD). By transforming structured and unstructured data into actionable insights, companies can launch products that better serve customer needs and quickly monetize trends. However, 40 to 50 percent of new products fail—a strikingly high figure that hasn’t changed much over the past 20 years despite an exponentially increasing volume of collected data [1]. How can companies better implement big data throughout the NPD process?

I was curious how the aforementioned new product fail rate differed among companies with big data-embedded NPD versus those without, and I found a 2020 study [2] comparing the respective success rates, which were broken down by the seven stages of the NPD process. With the exception of commercialization, every stage of the NPD process saw a significantly higher success rate with big data implementation. I created the below graphic to visualize the results.

So, given these stark comparisons, why aren’t more companies implementing big data throughout the NPD process? I see the following adoption barriers:

• Customer relationship management: Companies are drowning in data, but raw customer data is meaningless until it’s converted into information, analyzed for actionable insights that further business goals and then implemented.
• Production turnaround time: “New products” don’t remain new for long, especially in fast-cycle industries such as tech and fashion. Identifying the next big trend isn’t helpful if a company can’t monetize that trend into a saleable product before the trend passes.
• Senior management resistance: Senior management’s emphasis on data-driven decision-making is shown to have a substantial effect on NPD success rates. According to a recent study, NPD projects led by managers who placed a high emphasis on data-driven decision-making had a success rate of 68% compared to 23% among those with a low emphasis [2].

Lastly, here are a few examples of how companies across industries have harnessed the power of big data throughout the NPD process:

• Idea development: Back in 2014, L’Oréal Paris noticed a spike in search terms related to ombre hair and quickly went to market with the first at-home ombre kit [3]. L’Oréal is now looking to video diaries as a relatively low-cost technology to glean customer insights in the moment of use. Similarly, computer hardware and electronics company Lenovo gleans customer insights through its own Talend big data platform, which simultaneously acquires hundreds of external datasets from third parties, social networking feeds and Application Programming Interfaces (APIs) [4].
• Product design/product testing: In the food industry, analyzing big data can alert companies to potential food quality or safety issues during the product development stages. Sensor-based technology can collect real-time data on location or attributes so that issues can be identified and resolved quickly [3].

Sources

[1] Jing Guo, Qingjin Peng, Liyan Zhang, Runhua Tan, Jianyu Zhang. (2020) Estimation of product success potential using product value. International Journal of Production Research 0:0, pages 1-17.

[2] Wang, Yufan, et al. “Does Big Data-Embedded New Product Development Influence Project Success? This study explores the extent to which big data-embedded new product development influences product success and highlights the importance of a data-driven culture.” Research-Technology Management, vol. 63, no. 4, July-Aug. 2020, p. 35+. Gale Academic OneFile.

[3] Jagtap, S., & Linh Nguyen, K. D. (2019). Improving the new product development using big data: A case study of a food company. British Food Journal, 121(11), 2835-2848.

[4] Tan, K. H., & Zhan, Y. (2017). Improving new product development using big data: a case study of an electronics company. R&D Management, 47(4), 570–582.