OpenStack VXLAN

VXLAN (Virtual Extensible LAN) is an overlay network protocol that encapsulates Layer 2 Ethernet frames inside UDP packets. OpenStack Neutron uses VXLAN to create isolated tenant networks over a shared physical network, solving the 4,096 VLAN ID limit by supporting up to 16 million unique network segments.

How VXLAN Works in OpenStack

Each VXLAN segment is identified by a 24-bit VNI (VXLAN Network Identifier). Neutron assigns a unique VNI to each tenant network. Traffic between VMs on different compute nodes is encapsulated in UDP (default port 4789) and sent over the physical network.

Prerequisites

Step 1: Configure the ML2 Plugin

Edit /etc/neutron/plugins/ml2/ml2_conf.ini on the controller:

[ml2]
type_drivers = flat,vlan,vxlan
tenant_network_types = vxlan
mechanism_drivers = openvswitch

[ml2_type_vxlan]
vni_ranges = 1:10000

The vni_ranges defines the pool of VNIs that Neutron can assign to tenant networks.

Step 2: Configure the OVS Agent

Edit /etc/neutron/plugins/ml2/openvswitch_agent.ini on every compute and network node:

[ovs]
local_ip = 10.0.1.11  # this node's tunnel endpoint IP

[agent]
tunnel_types = vxlan
l2_population = true

[securitygroup]
firewall_driver = openvswitch

Set local_ip to each node's IP on the tunnel/overlay network. This is the VTEP address.

Step 3: Set the Physical MTU

VXLAN adds a 50-byte header. If your VMs expect 1500 MTU, the physical network must support at least 1550:

# On every compute and network node
sudo ip link set ens192 mtu 9000  # jumbo frames recommended

Configure Neutron to advertise the correct MTU to tenants in /etc/neutron/neutron.conf:

[DEFAULT]
global_physnet_mtu = 9000
path_mtu = 9000

Tenant networks will automatically get MTU = path_mtu - 50 = 8950.

Step 4: Configure the L3 Agent

The L3 agent creates routers that route between VXLAN tenant networks and external flat/VLAN networks. Edit /etc/neutron/l3_agent.ini:

[DEFAULT]
interface_driver = openvswitch
external_network_bridge =

Step 5: Restart Services

sudo systemctl restart neutron-server
sudo systemctl restart neutron-openvswitch-agent  # on all nodes
sudo systemctl restart neutron-l3-agent
sudo systemctl restart neutron-dhcp-agent

Step 6: Create a VXLAN Tenant Network

source openrc admin admin

openstack network create --provider-network-type vxlan tenant-net
openstack subnet create --network tenant-net \
  --subnet-range 192.168.1.0/24 \
  --dns-nameserver 8.8.8.8 tenant-subnet

Verify the VNI assignment:

openstack network show tenant-net -c provider:segmentation_id

Step 7: Verify VXLAN Tunnels

On a compute node, check that OVS has established VXLAN tunnels:

sudo ovs-vsctl show

You should see tunnel ports like:

Port vxlan-0a000112
    Interface vxlan-0a000112
        type: vxlan
        options: {df_default=true, in_key=flow, local_ip="10.0.1.11", out_key=flow, remote_ip="10.0.1.12"}

VXLAN vs VLAN vs Geneve

Geneve is the default for OVN deployments and is gradually replacing VXLAN in newer OpenStack installations.

Troubleshooting

Summary

VXLAN enables massive multi-tenant network isolation in OpenStack by overlaying Layer 2 segments on a Layer 3 physical fabric. Proper MTU configuration is the most critical detail for a successful deployment.