
Architecting for AI agility: How hybrid by design can help tech architectures accelerate business outcomes
- This is the third in a series of reports on how to design and implement your organization’s “Great Tech Reset,” using a method we call hybrid by design. In this installment: how hybrid by design technical architectures accelerate business outcomes.
Flashy acquisitions and headline-grabbing initiatives dominate corporate narratives, but a quieter revolution is brewing behind the scenes. Traditionally relegated to IT specialists, technology architecture is emerging as a strategic cornerstone for CEOs seeking a sustained competitive edge. We’re not talking about the latest trendy programming language. It’s about harnessing the power of technology to meet core objectives that define success: efficiency, scalability, and agility.
As gen AI proliferates, C-suites are embracing its promise. Sixty-two percent of business leaders believe generative AI is more reality than hype—up from only 33% a year ago. And generative AI has boosted overall AI ROI from 13% in 2022 to 31% today. While 31% may seem high at this stage in the AI game, organizations that have the highest maturity in gen AI capabilities are focusing on only a few high-value projects (those most likely to be successful) so it makes sense that early ROI is substantial.
But for many organizations, it’s not always smooth sailing. As they move to broader and deeper applications of gen AI—and begin to scale—they run into challenges. Specifically, they often overlook a key obstacle to ROI: their current technology architecture.
Thirty-eight percent of organizations think they have already designed the tech architecture needed to implement AI business solutions at enterprise scale. Based on IBM consulting experience, however, 38% appears high; these organizations may be underestimating what they’ll need from their architecture to scale gen AI.
Beyond jargon: A tech primer
Tech execs speak the language of technology, often discussing how hybrid cloud accelerates AI; nuances of architecture, infrastructure, and application layers, and other tech terms with ease. But sometimes their business counterparts are confused by—or uninterested in—the technical jargon. Fifty-five percent of business leaders say tech architecture is poorly understood by business stakeholders—not an ideal situation when you want to use IT architecture as a business advantage. So, to use an analogy, imagine a restaurant:
- Technical architecture is the blueprint of the restaurant: It defines how the kitchen is laid out, where the tables go, how the electricity gets routed to power the ovens. It ensures everything is arranged efficiently and works together smoothly.
- Technical infrastructure is the foundation of the restaurant: the building itself, the electrical wiring, the plumbing. It’s the basic equipment needed for anything to function. Without a solid foundation, the restaurant couldn’t operate.
- Enterprise platforms are the pre-built prep areas within the kitchen. Some are designed for pastries and desserts, others for main courses. And each area can be customized with additional features (specialty mixers, pizza ovens, etc.).
- Applications are the tools the chefs use: the ovens, refrigerators, plates, menus. They allow you to accomplish tasks, like taking orders or building products.
The difference hybrid-by-design architecture makes
The biggest issue today is that most technical architectures are fit for a bygone era; they were not designed with intention to serve today’s—let alone tomorrow’s—very digital, very interconnected business needs. For example, they’re not as modular and composable as current architectures, so they lock users into certain ways of working, slowing down product development.
Generative AI is a good example of a technology today’s architectures weren’t designed to support. Traditional architecture inhibits, rather than optimizes, what gen AI can do. Information is locked away in isolated databases, starving generative models of the rich fuel they need to learn and create. Fragmented workflows slow down the training and deployment of generative AI models. And outdated processors bottleneck the power needed to unleash gen AI’s full potential. It’s akin to a blueprint for a building before modern air conditioning systems, shared workspaces, and today’s power consumption needs. The HVAC, wiring, and overall design aren’t a good fit for the current business environment and ways of working.
But hybrid-by-design tech architecture lays the groundwork for the kind of long-term growth C-suites and boards expect. They deliberately ensure systems work together seamlessly, can scale for growth, and are built to last.
Download report translations
Originally published 15 July 2024
Well-defined architecture ensures that an organization’s technology aligns with its core business goals. Whether it’s driving productivity, improving experience, or enabling innovation, the right architecture provides flexibility and scalability. Think of a retail company aiming to provide seamless omnichannel experience. Robust architecture allows them to build apps that integrate their online store, physical locations, inventory management systems, promotions, customer service experiences, and other touchpoints, enabling a unified customer journey.
For many enterprises, architecture only hits C-suite conversations as gen AI takes hold. Why? Because it’s the thing that’s going to stop your progress. Seventy percent of business leaders say their tech architecture creates confusion, conflict, and disagreement. Yet 65% also say their tech architecture is critical to how they use IT to improve business performance. It’s a paradox.
Business leaders say the top two obstacles to successful tech architecture are:
- Difficulty explaining the impact of architectural decisions to business stakeholders (54%)
- Lack of understanding of business problems among architects (53%).
Strong tech architecture bridges this gap by creating a shared language between business and IT. By defining the technology roadmap and aligning it with business goals, architecture fosters collaboration where both sides work together to achieve objectives. As business needs change, architecture flexes so that it drives to the right outcomes at the right time.
Benign neglect isn’t benign
Imagine you’re building a cutting-edge restaurant. You’ve got ambitious architects and the most innovative materials, but you’re using a foundation from a rickety shed. That’s what neglecting technical architecture is like for generative AI. It’s building a future-changing tech marvel on a base that’s going to crumble at the first gust of innovation. Generative AI needs a firehose of information to work its magic. But legacy architectures are the equivalent to siloed data locked in virtual filing cabinets; they prohibit gen AI from reaching its full potential. In addition, many legacy systems are riddled with security holes—the equivalent of rolling out a welcome mat for hackers.

What you need to do
Re-architect your architecture
Architectures built without a clear business vision are often brittle and prone to failure. They struggle to adapt to changing needs and demands. The question isn’t: “Should we tweak the architecture?” It’s: “Are we bold enough to rebuild it for the future?”
Architecture by design means designing around business needs, not the hottest new technology. The new tech is just a tool to help get you to the business outcomes. But many organizations don’t use that script. Here’s how to begin to flip it:
- Develop a library of business-driven architecture patterns. Not every business problem requires its own unique architecture solution. Imagine a world where your team isn’t reinventing the architecture wheel (or the API) every time they face a business challenge. Developing a library of patterns can help teams work faster as they create architectures that address various aspects of the business. Pre-defined patterns accelerate development by providing reusable building blocks. It will also help eliminate the temptation to build with a tech focus versus a business-outcome focus. The library itself becomes a breeding ground for innovation. New patterns emerge by combining existing ones, leading to creative solutions for complex business challenges.
- Go data agnostic. As you create new architectures, your data strategy needs to keep pace to gain full benefits from it. Data agnosticism allows you to leverage the strengths of different architectural patterns for specific data types, increasing the speed of your data-driven decision making. Modern businesses rely on a variety of data sources: structured, unstructured, real-time, and historical. Evolving your data strategy to mesh with your architecture allows you to leverage different patterns for different data types, maximizing the value you extract from all your data assets. For each architecture pattern, identify the data patterns that are best suited for that pattern. For example, monolithic architecture might work best for structured data for efficient storage and retrieval. Microservices could handle real-time data for event-driven communication. Event-driven architecture works for unstructured data for event processing and analysis. And serverless architecture could be used for historical data for batch processing and analytics.
- Automate your architecture. The future belongs to architects who can leverage AI to build so they can imagine. We’re seeing the effect that gen AI can have on software development. Developers are freed from repeatable coding tasks and can pursue higher-order design product design work. The same kind of “assistant” use case works for architects. Engage tech architects in training small-model gen AI on the massive amount of architectural documentation your enterprise creates. When the AI assistant can generate routine architectural documentation, architects can spend more time improving customer experiences. Embracing AI doesn’t diminish your role as an architect; it amplifies it. Focus on the strategic aspects of architecture—designing systems that not only function flawlessly but also propel your business towards future success.
Sixty percent of executives say the way they design, deliver, and manage cloud architecture will require significant to radical changes within two years. And two-thirds say the same about infrastructure.
Navigating hybrid architecture requires careful orchestration. Streamlining data movement between on-premise and cloud environments, along with robust security protocols, is crucial. However, potential rewards—faster innovation cycles, cost optimization, and a future-proofed AI strategy—far outweigh challenges.
By intentionally designing a hybrid architecture, businesses can unlock the true potential of gen AI, accelerating innovation and securing a winning position in the AI-powered future. In fact, 88% of executives say that generative AI already accelerates innovation in their industry. With generative AI set to permeate every function and activity of the enterprise, it will soon materially impact business model innovation.
Over the next three years, executives report that AI and generative AI will support business model innovation in myriad ways: by providing access to additional data (88%), generating new insights from existing data (86%), expanding access to new markets (85%), and accelerating product and services development (84%). Given this, it’s easy to understand why CEOs cite business model innovation as their organization’s top challenge over the same timeframe.
Generative AI needs a powerful engine for raw processing power, such as on-prem processing capability, and the agility of cloud for rapid scaling and data access. Hybrid-by-design architecture delivers both, unlocking several key advantages for businesses:
- Rocket-fueled product-led development: Hybrid by design allows you to train core AI models on-premise, ensuring data privacy, while leveraging the cloud for rapid deployment of new features and A/B testing based on user feedback. This rapid iteration cycle fuels product-led development, with real-time user data, a key driver of growth. Imagine a retailer revamps its mobile app with a focus on product-led development powered by AI. The app utilizes a core AI model trained on-prem with anonymized customer data. As customers browse the app or physical stores, the AI analyzes their behavior in real-time, offering personalized product recommendations. The AI generates dynamic shopping lists based on a customer’s needs and past purchases. Additionally, the AI can present targeted promotions and coupons for relevant products, driving impulse purchases and increasing average order value.
- The power of choice: The biggest benefit of intentional hybrid architecture is flexibility. Businesses can choose the optimal environment for each stage of the AI lifecycle. Need to train a massive language model? Tap into your on-premise powerhouse. Experimenting with a new computer vision application? The cloud’s your better bet. It’s like having a private supercomputer for core tasks, and a limitless playground in the cloud to test and refine your AI ideas at warp speed. This agility gets your AI innovations to market faster, giving you a first-mover advantage.
- Security first: Hybrid architectures help businesses keep sensitive data and core applications safely guarded within their own walls, while at the same time leveraging the cloud’s robust security features for additional protection. This layered approach minimizes risk and fosters trust with customers and regulators. Think of it as a double layer of defense, keeping your core AI models under lock and key, while the cloud scrutinizes every update before it goes live.

What you need to do
Build the launchpad for gen AI
As you continue the countdown to launching generative AI at scale—you’ll need to ensure your launchpad is fit for the task. That means building or re-building your architecture so it is up to the challenge.
Tech leaders need to work with the business to ensure the most business-critical priorities for your architecture are well-defined and clear. Then define the architecture using a deliberate hybrid-by-design approach that will deliver on those priorities. More specific actions that will help your teams hit the mark:
- Codify and enforce consistent architectural principles. Architecture becomes real through hundreds of day-to-day decisions, so define a complete framework of architectural principles as well as the governance required to make and enforce implementation decisions. Without clear architectural principles, teams risk creating a patchwork of technologies leading to inefficiencies, security vulnerabilities, and maintenance nightmares. Gen AI architecture requirements should be meshed with decisions about security, data sharing, and development platforms. Zooming out a bit, those technology decisions also need to be in sync with decisions about integration and product-led design, which need to be aligned with overall business objectives.
- Don’t let vendors hijack your architecture. Some vendors will offer short-term cost cuts in return for a bigger share of digital workloads. Make your best deal with strategic providers, but don’t surrender the architectural control needed to deliver critical business outcomes. A robust architectural framework ensures every technology decision aligns with your core business objectives, not just a cloud provider’s sales goals. This focus on value ensures your AI investments drive tangible business results.
- Whittle down objectives to a critical few. When the business can’t articulate a few critical priorities, architects can’t design the best architectural blueprints. Don’t dilute the power of your architecture by trying to address every wish list item. Ensure business and IT teams work together to define the architecture required so they can deliver on the enterprise’s few highest-priority objectives.
Successful platforms unite the right data, model architectures, governance, and computing infrastructure to create value. As businesses become increasingly platform-based, they need a modern IT architecture. Generative AI could be the missing ingredient. It instills platform superpowers by transforming functions across the organization.
There’s a reason enterprise platforms are back in the spotlight. This time around, we’re witnessing a new generation of platforms specifically designed to address the challenges of today’s digital landscape and the demands of artificial intelligence (gen AI). Going back to our restaurant analogy, if enterprise platforms are prebuilt prep areas, they need to be built for the right purpose. A pastry corner wouldn’t help someone trying to make an Asian fusion dish.
Put another way: Traditional platforms tended to be somewhat generic solutions. Today’s hybrid-by-design enterprise platforms are tailored for the specific requirements of AI integration, digital transformation initiatives, and contemporary business needs.
Future-proofing with hybrid by design now brings multiple payoffs later. Platforms can bring a variety of benefits but the difference in hybrid by design is that it designs with this endgame in mind, building the platform around specific business needs.
Hybrid-by-design platforms allow internal teams and external partners to collaborate and experiment with ease in a way hybrid-by-default platforms can’t. This fosters a culture of innovation and accelerates development cycles. Gone are the days of clunky integrations. Modern enterprise platforms prioritize smooth collaboration with business partners, streamlining workflows and communication. And, by fostering innovation and streamlining processes, these platforms empower businesses to deliver a superior customer experience.
User experience soars—and so can gen AI user adoption
Slick-yet-simple interfaces hide complex choices—that’s the magic of a well-designed enterprise platform.
Gen AI models are powerful, but their inner workings can be opaque. Enterprise platforms provide user-friendly interfaces that translate complex AI functionalities into actionable insights and workflows. Business users don’t need to be data scientists to leverage the power of AI. A good technical architecture is designed to support business users in what they’re trying to do without making it a cumbersome, time-consuming process.
Platforms come pre-loaded with industry-specific tools and templates, ready to snap together and unleash your next big AI win. They’re a modular masterpiece, allowing IT to focus on innovation, not integration headaches.
Enterprise platforms also connect seamlessly with your existing data sources; it’s like having all the ingredients prepped and measured before you cook a meal. This cuts out data drudgery—no more manual collection. It also ensures clean, consistent data—the secret sauce for training powerful AI models.
Security, scalability, and governance for guardrails that flex
Beyond data magic and the ease modularity brings, platforms bring more peace of mind around cybersecurity issues. Gen AI is the new attack surface—and it’s vulnerable. Yet, gen AI adoption is outpacing trusted security approaches. Alarmingly, only 24% of gen AI projects are being secured. Many organizations are discovering this at the most inopportune time—after they’ve experienced a security breach.
Enterprise platforms build security in so your product teams don’t have to worry about it; they can focus on their business goals instead. They can eliminate the security risks siloed tools bring. They also force data integration and common governance.
Platforms also prioritize data security and compliance with industry regulations, mitigating the risks associated with handling sensitive data within gen AI workflows.
Another built-in advantage to platforms is the ability to scale quickly, especially in today’s environment. You don’t want your AI to outgrow its training ground. Enterprise platforms handle growing data demands and processing needs, so your AI can keep getting smarter. Plus, built-in governance tools keep everyone on the same page and ensure responsible AI development.
In essence, enterprise platforms bridge the gap between technical complexities and the business needs of everyday users. They empower business users to leverage the power of AI and other technologies without requiring extensive technical expertise, ultimately accelerating innovation and driving business value.
- Platforms provide security but different platforms handle it in different ways. An identity fabric is one of the more secure ways to address potential threats throughout your architecture. In keeping with hybrid-by-design principles, it’s an intentional and deliberate approach to baking security into your enterprise IT.
- Identity fabrics act as a central nervous system for managing user identities and access controls across an organization. They integrate various identity management tools and resources like directories, access control systems, and multi-factor authentication (MFA) into a single, unified platform. Imagine weaving together different threads (identity management systems) into a cohesive fabric.
- Benefits include:
- Central control: Identity fabrics also provide a central point to manage user identities, access rights, and permissions. They function as a control center where you can manage everything related to user access.
- Seamless access: With a unified system, users can access all authorized applications and resources with a single sign-on (SSO), improving user experience and reducing the need for multiple logins. Imagine using a single key (credential) to unlock (access) any door in your house (applications) while also restricting access (permissions) for the gardener who should only have access to the backyard (least privilege).
- Enhanced security: Identity fabrics strengthen security by enforcing consistent access control policies across the organization. This reduces the risk of unauthorized access and simplifies security audits. Think of it as having a single, strong security system for your entire house instead of individual locks on each door.
- Flexibility: An identity fabric can integrate with various cloud platforms, on-premise systems, and legacy applications, making it adaptable to diverse IT estates. Imagine your security system working seamlessly with all the different devices and appliances in your house.
- Scalability: As your organization grows, the identity fabric can scale to accommodate new users, applications, and access needs. Imagine easily adding new locks and security features to your house as your family expands.
- Here are some of the key benefits of using an identity fabric:
- Improved security posture: Consistent access control and centralized management reduce security vulnerabilities.
- Simplified user management: Easier provisioning, de-provisioning, and management of user identities.
- Enhanced user experience: Single sign-on eliminates login fatigue and improves user productivity.
- Reduced costs: Centralized management streamlines operations and potentially reduces licensing costs for multiple identity management tools.
- Increased agility: Easier integration with new technologies and faster onboarding of new applications.
- Overall, an identity fabric is a powerful tool for organizations looking to streamline identity and access management, improve security, and simplify IT operations in a complex and ever-evolving IT landscape.

What you need to do
Rebrand IT as a platform services provider
High demand for new tech initiatives can result in by-default architectures. Use platform engineering to embed business-driven, intentional architectural principles and a common user experience into platform design.
- Build platforms for tomorrow. Platform engineering lets you jettison generic architectures and embed business-driven design right from the start. This means a common, intuitive experience for users, and a foundation built to handle cutting-edge AI, app modernization, and even edge computing. One platform can handle all your needs, from the cutting edge to core business functions, all while keeping things user-friendly.
- Bake security in; don’t bolt it on. Design security directly into development and AI platforms. Imagine an identity fabric seamlessly woven into your platform, not a security blanket you throw on later. This platform-centric security approach secures applications from the ground up, so you can focus on innovation without security headaches.
- Unleash your developers by building an enterprise API catalog. APIs run on platforms and allow two computer systems to securely exchange information over the internet, extending the functionality of a platform or service to a larger audience. A catalog makes it easier for developers to find the specific APIs they need within the organization. Think of it like a central nervous system for your platform. Developers can find the tools they need instantly, no more time wasted sifting through docs. This supercharges innovation and gets your AI features out the door faster. It’s like giving your developers a map to the treasure trove of functionality hidden within your platform. Gen AI could even assist in its creation.
Hybrid by design in action
- The challenge
Audi needed to create a stable, scalable environment for innovative development. This required them to provision project environments faster to be able to build, deliver, and scale diagnostics, data management, and other projects across clouds. Audi also sought to reduce risks and remove dependencies with a flexible, modular architecture that could support iterative work. - The solution
Audi created a new as-a-service development environment based on Red Hat OpenShift, supplying Audi’s platforms, applications, and projects with a secure, stable, central environment for innovate development at scale. - The outcome
Audi reduced time to market by up to six months. With a common foundation, developers were able to work more efficiently to create, deliver, and migrate solutions across on-premise and cloud environments. Application scalability improved to meet demand.
- The challenge
- A fintech startup headquartered in Sweden, Edger Finance, aims to be the go-to solution that investors can use to navigate the stock market and make better investment decisions.
- In 2023, Edger—an IBM business partner and a member of the IBM Fintechx program—began collaborating with IBM’s client engineering and innovation studio teams to strengthen its processes and platform by piloting generative AI. The collaboration resulted in the creation of three AI-assisted processes that are offered in Swedish and English and were explored during a four-week minimum viable product (MVP) pilot:
- The first accelerates and simplifies the creation of a CEO summary from corporations’ quarterly reports.
- The second automates the extraction of data points that are within each report.
- The third allows investors to interact with the data in the report through a question-answer chat flow.
- Each assistant relies on IBM watsonx.ai™, an integrated suite of AI tools designed for security-rich, collaborative data management and process automation. The third assistant also utilizes IBM watsonx™ Assistant, a conversational AI platform that delivers automated self-service support.
- The tests conducted during the pilot demonstrated clear results and great potential for generative AI at Edger:
- 90% improvement in the turnaround time for quarterly report data extracts. Whereas previously the process could take up to a week, the pilot demonstrated that it could be accomplished in just four hours.
- Approximately 96% improvement in the time it takes to summarize the main points of a 30-page (or more) report. Whereas previously it could take an analyst up to half an hour to complete this task, the pilot demonstrated that it could be accomplished in a matter of seconds.
- The pilot also pointed toward several potential benefits for retail investors, such as greater efficiency in collecting and reviewing investment data as well as improved relevance and personalization of information provided to each investor utilizing the platform. By opening up possibilities for investors to interact with each report, Edger has made it easier for them to find, analyze and take action on information that is the most relevant to them and their investment strategy.
- Instead of succumbing to the complexity of tech sprawl so many global organizations experience, IBM embarked on a journey of simplification, transforming its IT architecture into a powerful engine for innovation. This story isn’t just about IBM; it’s a potential blueprint for any C-suite leader battling IT sprawl.
- The problem
Disparate IT environments—Z Systems®, cloud platforms, on-premise workloads, and an increasing number of edge devices—had over time become the norm. - Managing this complexity becomes increasingly unwieldy over time. Throwing people at the problem is what many companies do but it isn’t a strategic solution for this challenge. Automation is the key.
- The solution
The solution started with a common automation strategy. By automating provisioning, installation, and operation across all platforms, IBM could finally tame the beast. This automation layer has become the bedrock for further innovation. - Next came containerization. By recognizing that most workloads shared similar patterns, IBM containerized them, creating a standardized approach that transcended individual platforms. This not only simplified management but also paved the way for a more agile future.
- Data and applications
No IT architecture is complete without a data strategy. IBM addressed data at rest (databases), data in motion (application communication), and unstructured data, the lifeblood of AI. A robust data strategy ensures information is readily available and fuels the next chapter: applications. Applications are the face of the enterprise. The containerized and automated environment provides the ideal platform for both custom and off-the-shelf applications. This focus on applications underscores the ultimate goal: delivering business value. - Benefits for IBMers
The benefits of simplification resonate across the organization: - Decision-makers: Standardization, easier management, and faster time-to-value paint a compelling picture for leaders justifying IT investments.
- Engineers: Automation frees developers to focus on what machines can’t—creativity, problem-solving, and strategic thinking.
- Operators: A streamlined environment empowers operations to focus on higher-level tasks and leverage analytics for root-cause analysis.
- Developers: The rockstars of the show
Empowering developers is crucial. IBM prioritizes developer tools, including a common continuous integration/continuous delivery (CI/CD) layer and AI-powered code assistants. This ensures developers can code efficiently and deliver features that meet customer needs. - There’s a strong partnership between AI and developers. AI, a powerful tool that thrives on structure. A standardized environment ensures data accessibility, critical for training and fine-tuning AI models. But it’s not AI solo, of course.
- In addition, security is woven into the technical foundation. From physical security to data protection, IBM integrates security best practices into every layer of the architecture.
- A platform for the future
By simplifying its IT architecture, IBM has created a platform for agility, innovation, and growth. While it all began with IT simplification, the business benefits are the real outcome.
Technical architecture is the blueprint for your digital world. It defines how software components interact, ensuring smooth data flow and a system that scales gracefully. Without it, you risk data silos, clunky user experiences, and a system that crumbles under pressure. A hybrid-by-design technical architecture is the invisible backbone that keeps your digital world humming, allowing you to innovate, adapt, and thrive. Without wisely designed tech architecture, it’s very hard to meet business goals at all, let alone as rapidly as today’s competitive landscape demands.
Don’t get grounded by a tech architecture stuck in the pre hybrid-by-design era. Remember, in the race for AI dominance, the unsung hero isn’t the flashy new AI solution, but the invisible foundation beneath it all.
The C-suite that prioritizes building a future-proof tech architecture will be the one who is also building a future-proof business.
Download report translations
Originally published 15 July 2024