How does AWS Elastic Load Balancer Work?

Summary:

• AWS Elastic Load Balancer distributes incoming traffic across multiple targets using listeners, target groups, and sophisticated routing algorithms to ensure high availability and fault tolerance.
• Four distinct load balancer types exist. Application Load Balancer operates at Layer 7, Network Load Balancer at Layer 4, Gateway Load Balancer at Layer 3, and the legacy Classic Load Balancer. Each is optimized for specific traffic patterns and protocols.
• Modern enhancements include zonal shift capabilities, dual-stack IPv6-only configurations, modern TLS 1.3 security policies, and refined cross-zone load balancing controls.
• Understanding health checks, connection draining, sticky sessions, and error debugging patterns separates junior implementations from production-grade architectures.

Every millisecond your application spends routing traffic inefficiently translates directly into degraded user experience and lost revenue. AWS Elastic Load Balancer sits at the critical junction between your users and your compute resources. It makes decisions that determine whether your architecture scales gracefully under pressure or collapses during peak demand.

Understanding how AWS Elastic Load Balancer works is not merely academic knowledge for System Design interviews. It represents foundational expertise that separates engineers who build resilient distributed systems from those who simply deploy services and hope for the best.

The following diagram illustrates the high-level architecture of how an Elastic Load Balancer distributes incoming client requests across multiple Availability Zones and target instances.

Types of AWS Elastic Load Balancers

AWS provides four distinct load balancer types, each engineered for specific traffic patterns and architectural requirements. The choice between these options fundamentally shapes your system’s latency characteristics, protocol support, and operational complexity. Senior engineers must understand not just what each type does, but when the trade-offs justify selecting one over another in production environments.

Application Load Balancer (ALB) operates at Layer 7 of the OSI model, making routing decisions based on HTTP and HTTPS request content. ALB excels at content-based routing, examining headers, paths, query strings, and host names to direct traffic to appropriate target groups. This capability enables sophisticated deployment patterns including blue-green deployments, canary releases, and microservices architectures where different URL paths route to entirely different backend services.

Network Load Balancer (NLB) functions at Layer 4, handling TCP, UDP, and TLS traffic with extreme efficiency. NLB can process millions of requests per second while maintaining ultra-low latencies, typically under 100 microseconds for connection establishment. The critical differentiator is NLB’s support for static IP addresses and Elastic IP assignment. This proves essential when clients require specific IP addresses on an allowlist or when integrating with on-premises firewalls.

Gateway Load Balancer (GWLB) operates at Layer 3, designed specifically for deploying, scaling, and managing third-party virtual network appliances. GWLB uses the GENEVE protocol to encapsulate traffic. This enables transparent insertion of security appliances like firewalls, intrusion detection systems, and deep packet inspection tools into your traffic flow without modifying application architecture.

Classic Load Balancer (CLB) represents the legacy option, supporting both Layer 4 and Layer 7 routing but lacking the advanced features of newer alternatives. AWS recommends migrating existing CLB deployments to ALB or NLB. CLB receives minimal feature updates and carries higher operational overhead for equivalent functionality.

Comparative analysis across load balancer types

Selecting the appropriate load balancer requires evaluating multiple dimensions simultaneously. The following comparison captures the critical metrics that influence architectural decisions in production environments.

After clarifying the fundamental differences between load balancer types, understanding the internal mechanics of request routing becomes essential for optimizing performance and debugging production issues.

How AWS Elastic Load Balancer routes traffic

The request routing process within AWS Elastic Load Balancer involves multiple coordinated components working in sequence. When a client initiates a connection, the load balancer evaluates the request against configured listeners, applies routing rules, selects an appropriate target, and manages the connection lifecycle through completion. Each stage introduces configuration options that directly impact latency, availability, and resource utilization.

Listeners and target groups

Listeners define the entry points for incoming traffic, specifying the protocol and port combinations the load balancer monitors. Each listener associates with one or more target groups through routing rules that determine how requests map to backend resources. This separation of concerns enables sophisticated traffic management patterns.

• Protocol flexibility: A single ALB can host listeners for HTTP on port 80, HTTPS on port 443, and custom ports for internal services simultaneously.
• Rule-based routing: ALB listeners support up to 100 rules per listener, each evaluating conditions like path patterns, HTTP headers, query strings, and source IP ranges.
• Default actions: Every listener requires a default action (forward, redirect, fixed-response, or authenticate) that handles requests matching no explicit rules.

Target groups contain the registered compute resources that receive forwarded traffic. Targets can include EC2 instances, IP addresses, Lambda functions, or (in the case of NLBs) Application Load Balancers for advanced chaining scenarios. The target group configuration specifies health check parameters, deregistration delay, and the routing algorithm used for target selection.

The following diagram shows the relationship between listeners, rules, and target groups in a typical ALB configuration.

Routing algorithms and flow distribution

AWS Elastic Load Balancer employs different algorithms depending on the load balancer type and configuration. Understanding these algorithms helps predict traffic distribution patterns and diagnose uneven load scenarios.

Round robin serves as the default algorithm for ALB, distributing requests sequentially across healthy targets. This approach works well when targets have equivalent capacity and request processing times remain consistent. Round robin can create imbalances when backend processing times vary significantly or when persistent connections skew the distribution.

Least outstanding requests became available for ALB in recent updates, routing new requests to the target with the fewest in-flight requests. This algorithm adapts dynamically to varying backend performance, automatically directing traffic away from slower instances without manual intervention.

Flow hash algorithm governs NLB target selection, using a hash of the connection tuple (source IP, source port, destination IP, destination port, protocol) to consistently route packets from the same flow to the same target. This consistency proves critical for stateful protocols and applications requiring connection affinity without application-layer session management.

Cross-zone load balancing and Availability Zone distribution

AWS Elastic Load Balancer distributes traffic across multiple Availability Zones to ensure fault tolerance. The specific behavior depends on cross-zone load balancing configuration. When enabled, the load balancer distributes requests evenly across all registered targets regardless of their Availability Zone placement. When disabled, each load balancer node distributes traffic only to targets within its own zone.

The implications for capacity planning are significant.

1. Cross-zone enabled: Traffic distributes evenly across all targets, simplifying capacity calculations but potentially increasing inter-AZ data transfer costs.
2. Cross-zone disabled: Each AZ receives traffic proportional to its DNS weight, requiring careful target count balancing across zones to prevent overload scenarios.
3. Hybrid approaches: NLB allows per-target-group cross-zone settings, enabling fine-grained control for different backend services.

Consider the following scenario. An ALB with cross-zone disabled serves three Availability Zones, each receiving 33% of traffic. If Zone A contains 2 targets while Zones B and C contain 10 targets each, Zone A targets receive dramatically higher load per instance. This misconfiguration frequently causes cascading failures during traffic spikes.

With the routing mechanics established, examining recent AWS enhancements reveals how the platform continues evolving to address modern architectural challenges.

Modern features and resilience enhancements

AWS continuously extends Elastic Load Balancer capabilities to address emerging requirements around resilience, security, and operational efficiency. Recent platform updates have introduced several capabilities that senior engineers should incorporate into their architectural patterns.

Zonal shift and AZ evacuation

Zonal shift enables rapid traffic evacuation from an impaired Availability Zone without modifying DNS records or target group configurations. When activated through the AWS Route 53 Application Recovery Controller, zonal shift redirects load balancer traffic away from the specified zone within seconds. This compares to the minutes required for traditional DNS-based failover.

This capability proves invaluable during partial AZ degradation scenarios where health checks pass but latency or error rates increase. Rather than waiting for health check failures to trigger automatic target removal, operators can proactively shift traffic while investigating the underlying issue.

Dual-stack IPv6 and IPv6-only configurations

AWS now supports IPv6-only Application Load Balancers, eliminating the requirement for IPv4 address allocation in environments fully migrated to IPv6. This configuration reduces costs associated with IPv4 address scarcity and simplifies network architecture for organizations committed to IPv6 adoption.

Dual-stack configurations remain available for transitional architectures, allowing the load balancer to accept both IPv4 and IPv6 client connections while communicating with targets over either protocol. The target group IP address type setting controls whether targets register with IPv4 or IPv6 addresses. This enables gradual backend migration independent of client-facing protocol support.

TLS 1.3 default and security policy updates

TLS 1.3 is the modern standard security policy for HTTPS listeners, providing improved handshake performance (single round-trip) and stronger cipher suites compared to TLS 1.2. The ELBSecurityPolicy-TLS13-1-2-2021-06 policy supports both TLS 1.3 and TLS 1.2 for backward compatibility while preferring the newer protocol when clients support it.

For environments requiring strict compliance, AWS offers TLS 1.3-only policies that reject connections from clients unable to negotiate the latest protocol version. This enforcement proves particularly relevant for internal services where client capabilities are controlled and legacy support is unnecessary.

The following diagram illustrates the TLS termination flow and certificate management within an ALB configuration.

Understanding these features provides the foundation for addressing the practical concerns of cost management and performance optimization in production deployments.

Cost structure and performance optimization

AWS Elastic Load Balancer pricing follows a consumption-based model that varies by load balancer type. Effective cost management requires understanding the billing dimensions and implementing architectural patterns that minimize unnecessary charges while maintaining performance requirements.

Pricing dimensions explained

ALB pricing combines hourly charges with Load Balancer Capacity Unit (LCU) consumption. Each LCU measures the maximum of four dimensions during an hour.

• New connections: 25 new connections per second
• Active connections: 3,000 active connections per minute
• Processed bytes: 1 GB per hour for EC2 targets, 0.4 GB for Lambda
• Rule evaluations: 1,000 rule evaluations per second

NLB uses Network Load Balancer Capacity Units (NLCU) with different dimension thresholds optimized for Layer 4 traffic patterns. GWLB pricing follows a similar model with Gateway Load Balancer Capacity Units (GWLCU) reflecting the overhead of GENEVE encapsulation and appliance integration.

Optimization strategies

Several architectural patterns reduce ELB costs without sacrificing availability. Consolidating multiple ALBs into a single load balancer with path-based routing eliminates duplicate hourly charges while maintaining logical separation between services. This approach works particularly well for microservices architectures where each service previously received its own load balancer.

Connection pooling and keep-alive configuration significantly impact new connection costs. Ensuring backend services support HTTP keep-alive and configuring appropriate idle timeout values prevents unnecessary connection churn that inflates LCU consumption.

For NLB deployments, disabling cross-zone load balancing eliminates inter-AZ data transfer charges. These charges can represent 30-40% of total costs in high-throughput scenarios. This optimization requires careful capacity planning to ensure each zone maintains sufficient target capacity independently.

With cost considerations addressed, examining common failure modes and debugging approaches prepares engineers for the operational realities of production load balancer management.

Troubleshooting common issues

Production ELB deployments encounter predictable failure patterns that manifest as elevated error rates, increased latency, or complete service unavailability. Systematic debugging approaches accelerate resolution and prevent recurrence.

HTTP 5xx error analysis

The load balancer generates distinct 5xx error codes depending on failure location and cause. Understanding these distinctions directs investigation efforts appropriately.

• 502 Bad Gateway: The target returned a malformed response or closed the connection before sending a complete response. Investigate target application logs and connection timeout settings.
• 503 Service Unavailable: No healthy targets exist in the target group. Verify health check configuration and target instance status.
• 504 Gateway Timeout: The target failed to respond within the configured idle timeout period. Increase timeout values or optimize backend processing time.

The ELB access logs provide detailed request-level information including target processing time, response codes, and connection details. Enabling access logging to S3 and querying with Amazon Athena enables rapid identification of error patterns across millions of requests.

Health check configuration

Misconfigured health checks cause both false positives (unhealthy targets marked healthy) and false negatives (healthy targets removed from rotation). The health check parameters require careful tuning based on application characteristics.

1. Interval and timeout: The timeout must be less than the interval. A 30-second interval with 5-second timeout provides reasonable detection speed without excessive probe traffic.
2. Healthy/unhealthy thresholds: Higher thresholds prevent flapping but delay detection. Production systems typically use 2 healthy and 3 unhealthy thresholds.
3. Health check path (Shallow checks): The endpoint should verify local process liveness and reachability. Avoid “deep” health checks that verify external database connectivity or downstream dependencies; if a shared database experiences a brief blip, deep health checks will cause the load balancer to mark all instances as unhealthy simultaneously, triggering a massive cascading failure.

The following diagram shows the health check state machine and transition conditions.

Connection draining and deregistration

When targets deregister (manually or through Auto Scaling), the deregistration delay setting controls how long the load balancer continues sending existing connections to the target while preventing new connections. Setting this value too low causes in-flight requests to fail. Setting it too high delays scaling operations and deployment rollouts.

For most web applications, a 30-60 second deregistration delay provides sufficient time for request completion. Long-polling applications or WebSocket connections may require values of 300 seconds or more to avoid disrupting established sessions.

Conclusion

AWS Elastic Load Balancer provides the foundational traffic distribution layer that enables scalable, fault-tolerant architectures on AWS. The choice between ALB, NLB, GWLB, and CLB depends on protocol requirements, latency sensitivity, and operational complexity tolerance. ALB serves most web application needs while NLB addresses high-throughput, low-latency scenarios. Mastering the configuration of listeners, target groups, health checks, and cross-zone settings transforms load balancing from a deployment checkbox into a strategic architectural component.

Recent platform updates around zonal shift, IPv6-only support, and TLS 1.3 defaults reflect AWS’s continued investment in resilience and security capabilities. Engineers preparing for System Design interviews should demonstrate understanding of these features alongside the fundamental routing mechanics. Interviewers increasingly probe for awareness of recent platform evolution.

Build your mental model around the request lifecycle. This includes client connection, listener evaluation, rule matching, target selection, health verification, and response delivery. This framework provides the vocabulary and conceptual structure to reason about load balancer behavior under any scenario, from routine traffic distribution to complex failure mode analysis.