networking layer

---

INTRODUCTION   TO  ACI   Rene  Raeber   Dis+nguished  Engineer,  Datacenter  EMEAR   IEEE-­‐802.1  Architect  

Switzerland  

www.skatelescope.org  

Inspira=on  

The  Theore'cal  Minimum  

“What  You  Need  to  know  to  Start  Doing  Physics”   ISBN:  978-­‐1-­‐846-­‐14798-­‐2   Leonard  Susskind  &  George  Hrabovsky  

Applica=ons  &  Opera=ons  Drive  Infrastructure  Innova=on   The Next Wave

IT evolution and the triggers of change Opera'ons   Focus  

Virtualization

“Big-Iron”

Applica'ons   Development   Stack  &  Model  

x86 Rack-Mount

Blades Virtual Machines / Hypervisors

Client–Server

Mobile-Cloud Apps

Web 2.0

Fabric Computing (UCS)

Unified Fabric

Data  Center   Infrastructure  

DevOps

Full  Portability,   Cross-­‐PlaJorm  &   Automa'on  

Compute  &  Storage   Abstrac'on  

Workload   Unit    

BigData Analytics

Agility

Simplification

Compute  &   Storage  Access   Automa'on  

Network

LXC / Docker Containers

Distributed & Unstructured

Application Centric Infrastructure (ACI)

People

IoE

Network

Process

Things

LAN

SAN

Storage Access

Compute

Apps

Policy

InterCloud

Network  +  Services  Abstrac'on   &  Automa'on  

2008  

2009  

2014  

FUTURE

Data

WHAT IS ACI? OPEN RESTFUL APIS CENTRALIZED POLICY MODEL OPEN SOURCE

APIC

POLICY MODEL

CONTROLLER

NETWORK CONNECTS TO ALL COMPONENTS OF DATA CENTRE POLICY MODEL CONTROLS NETWORK AND INFORMATION FLOW

ACI

TWO TYPES OF LANGUAGE NETWORK LANGUAGE

APP LANGUAGE

VLAN

WEB

Subnets

APP

Bridging Routing

DB

IP Addresses Human Translator

APP-CENTRICITY FOR ACCESS CONTROL

WEB

APP

DB

CLEAR, SIMPLE DESCRIPTION OF HOW TIERS ARE ALLOWED TO COMMUNICATE

APP-CENTRICITY FOR SERVICE DEPLOYMENT

F/W ADC

WEB

ADC

APP

ANY SERVICE CAN BE ADDED BETWEEN TIERS

DB

DEFINING APPLICATION LOGIC TROUGH POLICY •  Contracts for Policy

Contracts are used to define relationships.

1 1

DEFINING APPLICATION LOGIC TROUGH POLICY •  Defining Provider Consumer Relationships

DB Farm

DEFINING APPLICATION LOGIC TROUGH POLICY •  Defining Provider Consumer Relationships

DB Farm

DEFINING APPLICATION LOGIC TROUGH POLICY •  Defining Provider / Consumer Relationships

DB Farm

POLICY ARCHITECTURE

THERE ARE TWO APPROACHES TO CONTROL SYSTEMS IMPERATIVE CONTROL

DECLARATIVE CONTROL

Baggage handlers follow sequences of simple, basic instructions

Air traffic control tells where to take off from, but not how to fly the plane

THERE ARE TWO APPROACHES TO CONTROL SYSTEMS IMPERATIVE CONTROL: VMWARE / NSX MODEL

DECLARATIVE CONTROL: ACI MODEL

Policy Manager + Control Plane

Policy Manager

WEB APP

APIC

DB SDN CONTROLLER

CENTRALIZED INTELLIGENCE

NO STANDARD PROTOCOL EXISTS

OVSDB FIREWALL

DATA PLANE

INTELLIGENCE DEPLOYED AT EVERY STAGE OF THE NETWORK

HYPERVISOR SWITCH

ADC

CONTROL + DATA PLANE

App, ops, and infrastructure requirements must be translated to network configuration

Automation / Speed

Simplification and acceleration from separating app, ops, and infrastructure

Controller becomes a bottleneck as it manages increasing state

Scalability and Resiliency

Distribute complexity to edges and operate disconnected from policy manager

App developers must still describe their requirements with low level constructs

Ease of Use

Self documenting policies automatically deployed or cleaned up from devices

Support for lowest common denominator feature set across vendors such as bridges, ports, and tunnels

Interoperability

Enables innovation through abstract policy without sacrificing interoperability

APP-CENTRICITY FOR TROUBLESHOOTING AND MONITORING HEALTH SCORE The Network Knows the App Structure and Components Easy to Follow Apps Around the DC Traditional 3-Tier Application

Visibility into the Health of the Infrastructure for the App APPLICATION NETWORK PROFILE

82% LATENCY

10 Microsecond(s) DROP COUNT

25 Packets Dropped VISIBILITY

APPLICATION NETWORK PROFILE

APPLICATION NETWORK PROFILE APPLICATION NETWORK PROFILE

VMs Servers

Ports Switches

Services Faults

LAYERS OF THE DATA CENTRE NETWORK

APPLICATION LAYER VIRTUALIZATION LAYER NETWORKING LAYER PHYSICAL LAYER

VIRTUALIZATION NETWORK VIRTUALIZATION IS WHAT SDN DOES Essentially re-inventing Networking Over Again – only virtual Important concepts: Location – Identity Separation and Central Point of Control Challenged by Scale, Reliability, Complexity, Visibility Must be done

VIRTUAL NETWORK NETWORK

CISCO AND VIRTUALIZATION COMPATIBILITY SCALEABILITY VISIBILITY APPLICATION LAYER VIRTUALIZATION LAYER

X86

NETWORKING LAYER PHYSICAL LAYER

HYPERVISOR

HYPERVISOR

HYPERVISOR

PHYSICAL

CISCO AND VIRTUALIZATION COMPATIBILITY SCALABILITY

•  Penalty free •  1 million end-points •  64K Tenants

VISIBILITY APPLICATION LAYER VIRTUALIZATION LAYER NETWORKING LAYER PHYSICAL LAYER

•  No flooding •  100 to 200K ports •  Single network

CISCO AND VIRTUALIZATION COMPATIBILITY SCALEABILITY VISIBILITY APPLICATION LAYER VIRTUALIZATION LAYER NETWORKING LAYER PHYSICAL LAYER

TRADITIONAL NETWORK

? PACKET

ACI NETWORK Drops Bandwidth Location Latency APP NAME

PACKET

NETWORKING LAYER

Constant latency Lowest Cost Infrastructure (Least # of Fibre Optics) Scale Out Not Up

APPLICATION LAYER VIRTUALIZATION LAYER NETWORKING LAYER PHYSICAL LAYER

USE OF FAT TREE TOPOLOGY Every Leaf Connects With Every Switch

NETWORK LAYER INNOVATIONS APIC

Dynamic Load Balancing (+2X) Dynamic Flow Prioritization (+10X) Fast re-routing Multicast multipath Traffic matrix monitoring Apps Run Faster and Better Network is more robust and efficient

End to End Congestion Monitoring

Focus on an Application Response Time

APPLICATION LAYER

STANDARD PRIORITY

VIRTUALIZATION LAYER

F1

F2

F2

F2

Dynamic Flow Prioritization

NETWORKING LAYER PHYSICAL LAYER

F1

HIGH PRIORITY

Flowlet Switching FLOWLETS F1

F1

F2

F2

F2

TOPOLOGIES

Symmetric (No link failure)

4x40Gbps

32x10Gbps

Asymmetric (Link failure)

4x40Gbps

32x10Gbps

4x40Gbps

32x10Gbps

3x40Gbps

32x10Gbps

Job Completion Time (sec)

ASYMMETRIC TOPOLOGY (LINK FAILURE)

500 400 ~2x improvement

300 200 100

ECMP

0 0

5

10

15 20 25 Trial Number

30

35

40

NETWORKING LAYER USE OF FAT TREE TOPOLOGY Constant latency Lowest Cost Infrastructure (Least # of Fibre Optics) Scale Out Not Up

APPLICATION LAYER VIRTUALIZATION LAYER NETWORKING LAYER PHYSICAL LAYER

Every Leaf Connects With Every Switch

MERCHANT SILICON ALONE LEAVES ROOM FOR IMPROVEMENT

NO ROUTING AND OVERLAY NO VISIBILITY BUFFERING

INDUSTRY BASELINE

Trident 2

VXLAN Bridging

12 MB Buffer

Visibility per Port

Counters per Packet and Bytes

MERCHANT + DYNAMIC LOAD BALANCING FLOW PRIORITIZATION MULTICAST MULTIPATH INLINE HW OVERLAY DB FAST RE-ROUTE

CISCO

ASIC INNOVATIONS

+

INDUSTRY BASELINE

VXLAN Routing Normalized Forwarding VXLAN/NVGRE/VLAN

Incremental 40 MB Buffer, Active Queue Management

APPLICATION VISIBILITY Elephant Traps, VXLAN Aware Trace-route

Atomic End-end Latency Measurements

Trident 2

VXLAN Bridging

12 MB Buffer

Visibility per Port

Counters per Packet and Bytes

INNOVATIONS IN HARDWARE

APPLICATION LAYER VIRTUALIZATION LAYER NETWORKING LAYER PHYSICAL LAYER

PRICE

PERFORMANCE

COST STRUCTURE for 1G to 1/10GT and 10G to 40G migration 1/3 less ASICS

INDUSTRY LEADING PRICE / LINE CARD BANDWITH 1.92 Tbps per slot 100G ready

PORT DENSITY 20% HIGHER Non-Blocking Density

FLEXIBILITY Standalone and ACI Traditional networking OR Application Centric

INNOVATIONS IN HARDWARE MERCHANT+ ASIC APPROACH Innovation in Cisco ASICs

PROGRAMMABILITY POWER EFFICIENCY PORT DENSITY STATEHIGHER OF THEAPI ART JSON/XML 20%

PERFORMANCE PRICE

INDUSTRY COST STRUCTURE LEADING PRICE / LINE for 1G CARD to 1/10GT BANDWITH 1.92 and 10G Tbpstoper 40G slot 100G migration ready

BACKPLANE FREE Linux Container for 36 Port 40 Gig NonDESIGN Density customer apps blocking 15% greater power and cooling efficiency

NEXUS 9000 P RICE

P ERFORMANCE

P ORT DENSITY

P ROGRAMMABILITY P OWER EFFICIENCY

INNOVATIONS IN HARDWARE: BIDI OPTICS 40G BiDi Optics

40G

Over 10G Multimode Fibre

Saves

TENS OF MILLIONS OF DOLLARS When Upgrading to 40G Switch Infrastructure*

*90% of Data Centres globally. BiDi’s 100m reach covers >90% of the DC distribution links Paul Kolesar, CommScope, IEEE 802.3 Next Gen Optics Study Group, Sep 2011

APPLICATION POLICY INFRASTRUCTURE CONTROLLER (APIC)

Single API/ Open/ Restful XML/ JSON Application Centric

Reliable

APIC Scalable

ENABLES THE APPLICATION CENTRIC INFRASTUCTURE

NORMATIVE Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø  Ø 

ACI Application Centric Infrastructure APIC Application Policy Infrastructure Controller DFA Distributed Fabric Automation VDP Virtual Station Interface Discovery Protocol VXLAN - Virtual eXtensible Local Area Network VXLAN Segment - VXLAN Layer 2 overlay network over which VM’s communicate VXLAN Overlay Network - another term for VXLAN Segment VXLAN Gateway - an entity which forwards traffic between VXLAN and non-VXLAN environments VTEP - VXLAN Tunnel End Point - an entity which originates and/or terminates VXLAN tunnels VLAN - Virtual Local Area Network VM - Virtual Machine VNI - VXLAN Network Identifier (or VXLAN Segment ID) ACL - Access Control List ECMP - Equal Cost Multipath IGMP - Internet Group Management Protocol PIM - Protocol Independent Multicast SPB - Shortest Path Bridging ToR - Top of Rack TRILL - Transparent Interconnection of Lots of Links