• IBM Institute for Business Value
  • Hybrid by Design
  • Generative AI
  • Benchmarking
  • IBV blog
  • About the IBV
  • IBM Consulting home page
  • Dealing with the AI data dilemma

    The right approach to data integration, governance, and tools can help companies manage the complexities of merging disparate data sources—and deliver better business results.
    Dealing with the AI data dilemma
    The right approach to data integration, governance, and tools can help companies manage the complexities of merging disparate data sources—and deliver better business results.

    Artificial intelligence (AI) is rapidly being adopted and embedded into business and society. Early in the pandemic, IBM Institute for Business Value research found that 84% of all organizations expected to maintain or increase the level of organizational focus on AI, with nearly a third boosting their AI investments as a direct result of the pandemic.

    In a recent survey, AI joined the Internet of Things (IoT) and cloud computing as the top 3 technologies CEOs expect will most help deliver results. And 43% of IT professionals report that their company has accelerated their rollout of AI as a result of the COVID-19 pandemic.

    This translates into a compelling need for companies to build a strategic AI capability—from strategy to operating model to talent—and integrate that capability in the business.

    Nearly 1 in 3 organizations boosted their AI investments as a direct result of the pandemic.

    Still, many AI projects languish, even after a promising proof-of-concept (PoC) phase. 90% of companies have difficulty scaling AI across their enterprises. So, it’s not surprising that about half of AI projects fail.

    AI’s unique data challenges

    Why? In a word, data. More than half of those responsible for AI strategy acknowledge that they are not clear on what their AI data needs are.  And 39% of IT professionals report that analyzing data to build and scale trusted AI is the most difficult part of their organization’s AI journey, while 32% say that data complexity and silos are a top barrier to AI adoption.

    No wonder, then, that more than half of organizations cite data as the culprit for AI projects stalling, with data quality issues cited as the top factor (58%), followed by a lack of well-curated data (45%), and data governance issues (40%).  In fact, even general best practices for advanced data capabilities may prove insufficient for AI. While AI may only be one use case for data—albeit an important one— lessons learned need to be extended and expanded in the AI realm.

    Roughly 1 in 3 IT professionals say that data complexity and silos are a top barrier to AI adoption.

    AI has several specific data considerations that make a difference in practice:

    • AI scale: With AI, generally speaking, the more data available, the better the quality and accuracy of the results. So, the volume of data required for AI can be far higher than even for some advanced analytics.
    • AI speed: Data must be current to achieve the speed of response needed for some AI insights and for the best predictions. At times it can even be real-time or very close to it.
    • Data variety: AI results are often better, not just with more data, but with data that can add context. Yet an AI model’s outcome can be altered by manipulating the data—maliciously or unintentionally—or even just from generalized “data drift,” so making sure all data is handled properly becomes essential.
    • Data quality: AI is highly sensitive to data, making it critical that the data corresponds to the underlying reality. In some instances, AI pays close attention to spikes that other traditional analytic methods might consider anomalous, so accuracy is essential.
    • Human perspective: How humans regard data—including the biases in our own experiences and the opacity in the “black boxes” of our own brains—often affects how we use it. Data has meaning in context and must be viewed and understood that way. Without the proper context, it can be unwittingly misused or misinterpreted.

    Too often, companies are overwhelmed by these complexities, becoming mired in well-meaning but misaligned approaches to solving the data challenges. To meet AI’s unique data needs, an organization first needs a clear-eyed view of the business problems it is trying to solve, then a pragmatic approach to solving them.

    Needs matching: Selecting the right tool from the AI-analytics continuum

    Needs matching: Selecting the right tool from the AI-analytics continuum.

    Read the full report to learn how organizations can plan for the challenges of rigorous data preparation and the complexities of merging disparate data sources—and deliver better business results.


    Bookmark this report


    Meet the authors

    Michael Haydock

    Connect with author:


    , IBM Fellow, Vice President and Chief Scientist, IBM Consulting, IBM


    Steven Eliuk

    Connect with author:


    , Vice President, AI and Governance Automation, Global Chief Data Office (GCDO), IBM


    Susara van den Heever

    Connect with author:


    , Director, Cloud and Cognitive Software Expert Labs, IBM

    Originally published 18 June 2021

    Overview Annual report Corporate social responsibility Inclusion@IBM Financing Investor Newsroom Security, privacy & trust Senior leadership Careers with IBM Website Blog Publications Automotive Banking Consumer Goods Energy Government Healthcare Insurance Life Sciences Manufacturing Retail Telecommunications Travel Our strategic partners Find a partner Become a partner - Partner Plus Partner Plus log in IBM TechXChange Community LinkedIn X Instagram YouTube Subscription Center Participate in user experience research Podcasts United States — English Contact IBM Privacy Terms of use Accessibility