DMVPN Issue

I have some past experience with point to point GRE tunnels (100+
locations with a single hub). This is my first time with DMVPN and Im
having some odd issues.

Topology:
2 Hub locations (3825's)
6-12mbps (can be scaled up if needed)

114 Remote locations (2801's)
768kbps SDSL or full T1 per site

All sites on the AT&T backbone.

Tunnels are up and running from the remote sites to the main hubs in a
lab environment. The problems are as follows:
1. While each remote router will connect to both hubs, it will only keep
a security association with 1 router. The tunnels continue to work and
the dynamic tunnels come up and down as needed for site to site
communications, but its very odd for not to see SA's. Is this normal? If
so, thats fine but I would like to make sure I'm not missing something.

2. What would be the best way to connect the 2 disparate hubs? I can
drop in a 2801 and bring up a point to point GRE tunnel but I would
prefer to have that for failover and run the main connection off the 3825's.

I have attached (scrubbed) configs. The remote1 config would be for an
SDSL site.

HUB
---------------------------------
Current configuration : 1941 bytes
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname HUB1
!
boot-start-marker
boot-end-marker
!
!
no aaa new-model
!
resource policy
!
memory-size iomem 10
ip subnet-zero
ip cef
!
!
!
!
no ip domain lookup
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
username cisco privilege 15 password 0 cisco
!
!
!
crypto isakmp policy 1
encr 3des
authentication pre-share
group 2
crypto isakmp key dmvpnkey address 0.0.0.0 0.0.0.0
crypto isakmp keepalive 30 5
!
!
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
!
crypto ipsec profile IPSECPROF
set transform-set ESP-3DES-SHA
!
!
!
!
!
interface Tunnel199
bandwidth 1000
ip address 10.8.199.254 255.255.255.0
no ip redirects
ip mtu 1400
no ip next-hop-self eigrp 10
ip nhrp authentication dmvpnkey
ip nhrp map multicast dynamic
ip nhrp network-id 199
ip nhrp holdtime 600
ip tcp adjust-mss 1360
no ip split-horizon eigrp 10
delay 1000
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 199
tunnel protection ipsec profile IPSECPROF
!
interface GigabitEthernet0/0
Description Inside
ip address 10.8.253.254 255.255.255.0
ip nat inside
ip virtual-reassembly
duplex auto
speed auto
media-type rj45
negotiation auto
!
interface GigabitEthernet0/1
description Outside
ip address 12.1.1.106 255.255.255.248
ip nat outside
ip virtual-reassembly
duplex auto
speed auto
media-type rj45
negotiation auto
!
router eigrp 10
network 10.0.0.0
no auto-summary
!
ip classless
ip route 0.0.0.0 0.0.0.0 12.1.1.105
!
!
ip http server
no ip http secure-server
ip nat inside source list 110 interface GigabitEthernet0/1 overload
!
access-list 110 permit ip 10.8.253.0 0.0.0.255 any
!
!
!
!
control-plane
!
!
!
!
!
!
!
!
!
!
line con 0
stopbits 1
line aux 0
stopbits 1
line vty 0 4
login local
!
scheduler allocate 20000 1000
!
end
-------------------------
Hub2
-------------------------
Current configuration : 2000 bytes
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname HUB2
!
boot-start-marker
boot-end-marker
!
!
no aaa new-model
!
resource policy
!
memory-size iomem 10
ip subnet-zero
ip cef
!
!
!
!
no ip domain lookup
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
username cisco privilege 15 password 0 cisco
!
!
!
crypto isakmp policy 1
encr 3des
authentication pre-share
group 2
crypto isakmp key dmvpnkey address 0.0.0.0 0.0.0.0
crypto isakmp keepalive 30 5
!
!
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
!
crypto ipsec profile IPSECPROF
set transform-set ESP-3DES-SHA
!
!
!
!
!
interface Tunnel198
bandwidth 1000
ip address 10.8.198.254 255.255.255.0
no ip redirects
ip mtu 1400
no ip next-hop-self eigrp 10
ip nhrp authentication dmvpnkey
ip nhrp map multicast dynamic
ip nhrp network-id 198
ip nhrp holdtime 600
ip tcp adjust-mss 1360
no ip split-horizon eigrp 10
delay 1000
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 198
tunnel protection ipsec profile IPSECPROF
!
interface GigabitEthernet0/0
Description Inside
ip address 10.8.243.254 255.255.255.0
ip nat inside
ip virtual-reassembly
duplex auto
speed auto
media-type rj45
negotiation auto
!
interface GigabitEthernet0/1
description Outside
ip address 12.2.2.234 255.255.255.248
ip nat outside
ip virtual-reassembly
duplex auto
speed auto
media-type rj45
negotiation auto
!
router eigrp 10
network 10.0.0.0
no auto-summary
!
ip classless
ip route 0.0.0.0 0.0.0.0 12.2.2.233
!
!
ip http server
no ip http secure-server
ip nat inside source list 110 interface GigabitEthernet0/1 overload
!
access-list 110 permit ip 10.8.243.0 0.0.0.255 any
!
!
!
!
control-plane
!
!
!
!
!
!
!
!
!
!
line con 0
stopbits 1
line aux 0
stopbits 1
line vty 0 4
login local
!
scheduler allocate 20000 1000
!
end
-----------------------
Remote1
-----------------------
Current configuration : 2421 bytes
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname DENNY-VPN
!
boot-start-marker
boot-end-marker
!
!
no aaa new-model
!
resource policy
!
memory-size iomem 25
mmi polling-interval 60
no mmi auto-configure
no mmi pvc
mmi snmp-timeout 180
ip subnet-zero
ip cef
!
!
!
!
no ip domain lookup
!
!
!
!
username cisco privilege 15 password 0 cisco
!
!
!
crypto isakmp policy 1
encr 3des
authentication pre-share
group 2
crypto isakmp key dmvpnkey address 0.0.0.0 0.0.0.0
crypto isakmp keepalive 30 5
!
!
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
!
crypto ipsec profile IPSECPROF
set transform-set ESP-3DES-SHA
!
!
!
!
interface Tunnel198
bandwidth 1000
ip address 10.8.198.26 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication dmvpnkey
ip nhrp map multicast 12.2.2.234
ip nhrp map 10.8.198.254 12.2.2.234
ip nhrp network-id 198
ip nhrp holdtime 600
ip nhrp nhs 10.8.198.254
ip tcp adjust-mss 1360
delay 1000
tunnel source FastEthernet0/1
tunnel mode gre multipoint
tunnel key 198
tunnel protection ipsec profile IPSECPROF shared
!
interface Tunnel199
bandwidth 1000
ip address 10.8.199.26 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication dmvpnkey
ip nhrp map multicast 12.1.1.106
ip nhrp map 10.8.199.254 12.1.1.106
ip nhrp network-id 199
ip nhrp holdtime 600
ip nhrp nhs 10.8.199.254
ip tcp adjust-mss 1360
delay 1000
tunnel source FastEthernet0/1
tunnel mode gre multipoint
tunnel key 199
tunnel protection ipsec profile IPSECPROF shared
!
interface FastEthernet0/0
description Inside
ip address 10.8.26.254 255.255.255.0
ip nat inside
ip virtual-reassembly
ip route-cache flow
duplex auto
speed auto
!
interface FastEthernet0/1
description Outside
ip address 12.3.3.90 255.255.255.248
ip nat outside
ip virtual-reassembly
duplex auto
speed auto
!
router eigrp 10
passive-interface FastEthernet0/1
network 10.0.0.0
no auto-summary
eigrp stub connected
!
ip classless
ip route 0.0.0.0 0.0.0.0 12.3.3.89
!
ip http server
no ip http secure-server
ip nat inside source list 110 interface FastEthernet0/1 overload
!
access-list 110 permit ip 10.8.26.0 0.0.0.255 any
!
!
control-plane
!
!
line con 0
line aux 0
line vty 0 4
login local
!
scheduler allocate 20000 1000
end

Majsa

Not sure of the cause of your DMVPN problems but wanted to make your
aware of the follwoing 12.4 DMVPN bug,

CSCsc43989 Bug Details


Headline CEF adjacency inconsistent with NHRP cache entry
Product IOS
Feature OTHERS Duplicate of
Severity 3 Severity help Status Information Required Status help
First Found-in Version 12.4T All affected versions
First Fixed-in Version 12.4(7.15) Version help
Release Notes

Symptom: Packet forwarding issue to DMVPN spokes due to CEF adjacency
inconsistency with NHRP cache
information. From behind hub, users may not be able to reach (e.g.
ping) certain DMVPN spokes.

Workaround: Disable CEF on the hub.

Merv