Understanding AWS Infrastructure

Summary:

• AWS infrastructure spans 39 Regions and 123 Availability Zones, with specialized deployment options including Local Zones, Wavelength Zones, and Edge Locations that bring compute closer to end users.
• Purpose-built silicon like Trainium2 and Graviton4 processors deliver significant performance gains for AI workloads while reducing cost per inference by up to 50% compared to previous generations.
• The global network backbone encompasses over 9 million kilometers of fiber, including the new Fastnet subsea cable system designed to reduce transatlantic latency below 60 milliseconds.
• Sustainability innovations such as IRHX cooling technology and the shared responsibility model for security form critical pillars of modern AWS infrastructure design.

When a single millisecond of latency can cost an e-commerce platform millions in lost revenue, understanding the physical and logical architecture beneath your cloud workloads becomes a competitive advantage rather than academic knowledge. AWS infrastructure represents the largest and most sophisticated global cloud deployment in existence. Yet many engineers interact with it as an abstraction without grasping how Regions interconnect, why Availability Zones provide genuine fault isolation, or how purpose-built chips are reshaping the economics of AI inference. This guide dissects AWS global infrastructure from the physical data center layer through the networking backbone to the specialized deployment zones that enable sub-10-millisecond latency for edge applications.

The following diagram illustrates the hierarchical relationship between AWS infrastructure components, from global Regions down to individual Edge Locations.

Foundations of Regions, Availability Zones, and edge deployment

AWS infrastructure begins with the concept of geographic isolation through Regions, each representing a cluster of data centers in a specific geographic area. As of early 2025, AWS operates 39 Regions globally with 123 Availability Zones. The company has announced plans for additional Regions in New Zealand, Thailand, Taiwan, and the Kingdom of Saudi Arabia. Each Region operates as an independent failure domain with its own power, cooling, and networking infrastructure. A catastrophic event in one Region has no architectural impact on workloads running in another.

Within each Region, Availability Zones provide the fundamental unit of fault tolerance that distinguishes cloud architecture from traditional data center deployments. An Availability Zone consists of one or more discrete data centers with redundant power, networking, and connectivity. These zones are physically separated from other AZs in the same Region by meaningful distance (typically several kilometers) yet connected through dedicated high-bandwidth, low-latency fiber links.

This separation ensures that localized failures such as power grid issues, flooding, or fire affect only a single AZ. The synchronous replication capabilities between AZs enable recovery point objectives measured in seconds rather than hours.

Local Zones and Wavelength Zones for latency-sensitive workloads

Beyond the core Region and AZ structure, AWS extends its infrastructure footprint through specialized deployment options designed for workloads with stringent latency requirements. Local Zones place compute, storage, database, and other select AWS services closer to large population centers where no full Region exists. Currently, AWS operates over 30 Local Zones in metropolitan areas like Los Angeles, Boston, Houston, and Miami. These zones enable single-digit millisecond latency to end users in those markets.

Wavelength Zones take this concept further by embedding AWS compute and storage directly within telecommunications providers’ 5G networks. This architecture eliminates the network hops between the mobile device and the cloud, enabling latencies below 10 milliseconds for applications like real-time gaming, autonomous vehicle coordination, and industrial IoT. The key distinction between these deployment options can be summarized as follows:

• Local Zones: Extend Region services to metro areas, ideal for media rendering, real-time gaming backends, and hybrid deployments requiring low latency to on-premises systems.
• Wavelength Zones: Embedded in 5G networks for mobile edge computing, targeting connected vehicles, AR/VR streaming, and smart factory applications.
• Edge Locations: Over 600 points of presence globally for content delivery (CloudFront) and DNS resolution (Route 53), optimizing last-mile delivery rather than compute.

The following table provides a comparative analysis of AWS infrastructure deployment options, helping architects select the appropriate layer for specific use cases.

Understanding these infrastructure layers establishes the foundation for examining the compute, storage, and AI-optimized hardware that runs within them.

Core compute, storage, and AI-optimized hardware

The physical servers powering AWS infrastructure have evolved dramatically from commodity x86 hardware to a sophisticated mix of general-purpose processors, custom silicon, and purpose-built accelerators. This evolution reflects both the diversification of cloud workloads and AWS’s strategic investment in vertical integration to optimize price-performance ratios. At the compute layer, engineers must now navigate choices between Intel, AMD, AWS Graviton, and specialized AI chips, each with distinct performance characteristics and cost implications.

AWS Graviton processors represent the company’s most significant infrastructure innovation of the past five years. Built on Arm architecture, Graviton4 processors deliver up to 30% better compute performance compared to Graviton3 while improving energy efficiency. For workloads that can run on Arm-compatible software stacks (which now includes the vast majority of Linux-based applications), Graviton instances offer the best price-performance ratio in the AWS portfolio. The R8g, M8g, and C8g instance families powered by Graviton4 have become the default recommendation for new deployments where maximum compatibility is not required.

Purpose-built chips for AI and machine learning workloads

The explosion of generative AI and large language model deployments has driven AWS to develop specialized silicon that dramatically outperforms general-purpose GPUs for inference and training workloads. Showcased at AWS re:Invent 2024, the new Trainium2-powered instances offer 30 to 40% better price performance compared to GPU-based EC2 P5e and P5en instances. Alongside individual instances, AWS introduced the new EC2 Trn2 UltraServers, which feature 64 interconnected Trainium2 chips, offering up to 83.2 peak petaflops of compute. This architecture enables the training of massive trillion-parameter models, exemplified by Project Rainier—a colossal AI compute cluster featuring hundreds of thousands of interconnected Trainium2 chips that AWS is building specifically for Anthropic to train its future Claude models. Furthermore, AWS has already teased its 3-nanometer Trainium3 chips, expected in late 2025, promising to quadruple the performance of current UltraServers.

For inference workloads, the Inferentia2 chip family provides a compelling alternative to NVIDIA GPUs for production deployments. According to AWS ML infrastructure documentation, Inf2 instances deliver up to 50% lower cost per inference compared to comparable GPU instances for transformer-based models. The economic advantage becomes particularly pronounced at scale, where inference costs often dominate the total cost of ownership for AI applications.

Consider the following decision framework when selecting AI infrastructure:

1. Training large models (100B+ parameters): Trainium2 UltraServers or P5 instances with NVIDIA H100 GPUs for maximum flexibility.
2. Training medium models (1B-100B parameters): Trn1 instances offer the best price-performance for PyTorch and JAX workloads.
3. High-throughput inference: Inf2 instances for transformer models, with Graviton4 instances for traditional ML models.
4. Low-latency inference: Consider Local Zones with Inf2 or GPU instances to minimize network latency to end users.

The compute layer represents only one dimension of infrastructure optimization. The networking backbone that connects these resources globally determines the practical latency and throughput characteristics that applications experience.

Networking, latency, and global backbone

AWS operates one of the largest private networks on the planet, spanning over 9 million kilometers of fiber optic cable across terrestrial and subsea routes. This network backbone connects all AWS Regions, Availability Zones, and Edge Locations through a fully redundant mesh topology designed to survive multiple simultaneous cable cuts without service degradation. Unlike the public internet, where traffic traverses multiple autonomous systems with unpredictable latency and packet loss, traffic between AWS infrastructure components travels exclusively on AWS-controlled fiber with deterministic performance characteristics.

The most significant recent addition to this backbone is the Fastnet subsea cable system, announced in late 2024 and scheduled for completion in 2026. According to IT Pro reporting, Fastnet will provide a new transatlantic route connecting the eastern United States to Ireland and continental Europe with target latencies below 60 milliseconds. This investment addresses a critical bottleneck for enterprises running globally distributed applications that require synchronous data access across North America and Europe.

Network architecture within Availability Zones

Within data centers, AWS has developed proprietary networking technology that eliminates traditional bottlenecks in east-west traffic between instances. The Scalable Reliable Datagram (SRD) protocol, implemented in the Nitro networking cards present in all modern EC2 instances, provides reliable packet delivery with latency characteristics approaching raw UDP while maintaining the ordering guarantees applications expect. SRD enables the Elastic Fabric Adapter (EFA) technology that powers high-performance computing and distributed machine learning workloads.

For architects designing latency-sensitive systems, understanding the network topology within and between Availability Zones is essential. Key characteristics include:

• Intra-AZ latency: Typically sub-millisecond for instances in the same placement group, with 25-100 Gbps bandwidth depending on instance type.
• Inter-AZ latency: Consistently below 2 milliseconds within a Region, enabling synchronous database replication without significant performance penalty.
• Inter-Region latency: Varies by geographic distance, ranging from 20ms (US East to US West) to 150ms+ (US to Asia Pacific).

The following diagram illustrates how traffic flows through the AWS network backbone from an end user through Edge Locations to the origin infrastructure in a Region.

Network performance ultimately depends on the physical infrastructure housing these systems. The resilience, security, and sustainability characteristics of AWS data centers form the final critical layer of infrastructure understanding.

Resilience, security, and sustainability in AWS infrastructure

The physical security and environmental design of AWS data centers reflects decades of operational learning combined with substantial capital investment. Each facility implements multiple layers of physical security including perimeter fencing, professional security staff, biometric access controls, and 24/7 video surveillance. The more architecturally significant innovations occur in the power, cooling, and compute isolation systems that determine both reliability and environmental impact.

Power infrastructure at AWS data centers follows an N+1 or 2N redundancy model depending on the criticality tier. Utility power feeds from multiple independent substations, with on-site diesel generators providing backup capacity sufficient to run the entire facility indefinitely. Uninterruptible power supplies (UPS) bridge the gap between utility failure and generator startup, ensuring continuous operation through the transition. This layered approach delivers the 99.99% availability SLA that underpins AWS’s enterprise commitments.

The Nitro system and hardware-based security isolation

The AWS Nitro system represents the most significant security architecture innovation in cloud computing. By offloading virtualization, storage, and networking functions to dedicated hardware cards, Nitro eliminates the hypervisor attack surface that has historically concerned security teams evaluating multi-tenant cloud environments. The Nitro Security Chip provides a hardware root of trust that validates system firmware on every boot, preventing persistent malware from surviving instance restarts.

This hardware-based isolation enables features like Nitro Enclaves, which create cryptographically isolated compute environments for processing sensitive data. Within an enclave, even the instance owner cannot access memory or attach debuggers. This provides a trusted execution environment for use cases like payment processing, healthcare data analysis, and cryptographic key management.

The shared responsibility model for AWS infrastructure security divides obligations as follows:

• AWS responsibility: Physical security, hypervisor security, network infrastructure, and the Nitro hardware platform.
• Customer responsibility: Operating system patching, application security, identity management, and data encryption.
• Shared elements: Network configuration (security groups, NACLs), encryption key management, and compliance validation.

Sustainability innovations and environmental impact

AWS achieved its goal of matching 100% of the electricity consumed by its global operations with renewable energy in 2023, seven years ahead of its original 2030 target—and remains committed to achieving net-zero carbon by 2040. Beyond renewable energy procurement, the company has invested heavily in data center efficiency innovations that reduce the environmental footprint per unit of compute. The most notable of these is the Indirect Evaporative Cooling (IRHX) system deployed in newer facilities, which uses outside air and water evaporation rather than traditional compressor-based air conditioning.

According to AWS sustainability reports, IRHX technology reduces cooling energy consumption by up to 20% compared to conventional systems while eliminating the need for refrigerants with high global warming potential. In water-stressed regions, AWS has developed alternative cooling approaches, including direct liquid cooling for high-density AI compute racks. These investments reflect both environmental responsibility and economic optimization, as energy costs represent a significant portion of data center operating expenses.

Conclusion

AWS infrastructure has evolved from a simple virtual server offering into a globally distributed system of Regions, Availability Zones, and specialized edge deployments that enable architects to optimize for latency, cost, and resilience simultaneously. The key architectural insight is that these infrastructure layers are not merely geographic abstractions. They represent genuine physical isolation with distinct failure domains, networking characteristics, and service availability. Purpose-built silicon like Trainium2 and Graviton4 has shifted the economics of cloud computing, making AWS-specific hardware optimization a legitimate architectural consideration rather than vendor lock-in.

Looking ahead, the continued expansion of Local Zones and Wavelength Zones will blur the traditional boundary between cloud and edge computing. This enables new application architectures that were previously impossible due to latency constraints. The completion of the Fastnet subsea cable and similar backbone investments will further reduce the performance penalty of globally distributed deployments. For engineers preparing for System Design interviews or architecting production systems, deep familiarity with AWS infrastructure components provides the foundation for making informed trade-offs between availability, performance, and cost that distinguish senior practitioners from those who treat the cloud as an opaque abstraction.