[URGENT] cleaning vty session.

Router: Cisco 2611 - Version 12.3(15)

How to clean vty line sessions?

all commands like

clear line vty <#vty>

disconnect ssh vty <#vty>

disconnect ssh <#ssh session>

don't work....

don't tell me to reboot the router.

I have also deleted the keys (zeroize) to shutdown the ssh server but those session are still up.
The router is quite critical....
I also tried to clean the vty terminals while the the ssh server was down: nothing.

TIA for any hints or tips.

Alex.

 

I find this confusing too:-

There are two cases:-

1.
You have opened a telnet session /TO/ a router and want to
close it.


sh line

clear line n



2.
You have opened a telned sessions /FROM/ a router and
want to close it from the opiginating router.

sh sess

disconnect n



How to find which session is which I don't know.

 

All the session I opened are ssh sessions (5)

All my tentatives to close those ssh session were made using the console.

No results at all :-(, even with your tips (already tried)

Thanks, Alex

 

I made the perhaps rash assumption that telnet and ssh
were equivalent in the respect.

This pretty much MUST work.

Let me test it:-

SSH twice to a router

* 2 VTY - - - - 23 12 0 0/0
-
* 3 VTY - - - - 23 5 0 0/0
-

#sh tcp 3

tty3, virtual tty from host x.x.x.x
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 255
Local host: x.x.x.x, Local port: 22 ! ## NOTE TCP 22 - SSH


#clear line 3
[confirm]
[OK]

! My Putty window closed itself.

#sh line
* 2 VTY - - - - 23 12 0 0/0
-
3 VTY - - - - 23 5 0 0/0
-


OK it works here.

Why don't you post a session log
showing what it not working.

i.e.

sh line
sh tcp n
clear line n
sh line ! and we will see that the clear has failed.

You WILL need to be "Enabled".

 

Thanks bod43,

I'm still on the way.
Maybe the following output clarifies what's happening on that router:

----------------------------------------------------------------------------
Router-1#sh tcp vty 0

tty66, virtual tty from host aaa.eee.fff.ggg
Connection state is CLOSEWAIT, I/O status: 8, unread input bytes: 0
Local host: aaa.bbb.ddd.eee, Local port: 22
Foreign host: aaa.eee.fff.ggg, Foreign port: 52377

Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x13BBA2523):
Timer Starts Wakeups Next
Retrans 170 0 0x0
TimeWait 0 0 0x0
AckHold 182 1 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0

iss: 1133855515 snduna: 1133862255 sndnxt: 1133862255 sndwnd: 16092
irs: 3930970121 rcvnxt: 3930973946 rcvwnd: 4088 delrcvwnd: 40

SRTT: 300 ms, RTTO: 303 ms, RTV: 3 ms, KRTT: 0 ms
minRTT: 88 ms, maxRTT: 400 ms, ACK hold: 200 ms
Flags: passive open, higher precedence, retransmission timeout

TCB is waiting for TCP Process (3)

Datagrams (max data segment is 536 bytes):
Rcvd: 302 (out of order: 0), with data: 182, total data bytes: 3823
Sent: 210 (retransmit: 0, fastretransmit: 0), with data: 200, total data bytes: 6739
Router-1#sh tcp vty 1

tty67, virtual tty from host aaa.eee.fff.ggg
Connection state is ESTAB, I/O status: 1, unread input bytes: 140
Local host: aaa.bbb.ddd.eee, Local port: 22
Foreign host: aaa.eee.fff.ggg, Foreign port: 39263

Enqueued packets for retransmit: 0, input: 7 mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x13BBA4E78):
Timer Starts Wakeups Next
Retrans 77 0 0x0
TimeWait 0 0 0x0
AckHold 75 5 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0

iss: 2527521794 snduna: 2527524070 sndnxt: 2527524070 sndwnd: 16596
irs: 1581378661 rcvnxt: 1581380433 rcvwnd: 3988 delrcvwnd: 0

SRTT: 300 ms, RTTO: 303 ms, RTV: 3 ms, KRTT: 0 ms
minRTT: 52 ms, maxRTT: 300 ms, ACK hold: 200 ms
Flags: passive open, higher precedence, retransmission timeout

TCB is waiting for TCP Process (4)

Datagrams (max data segment is 536 bytes):
Rcvd: 127 (out of order: 2), with data: 79, total data bytes: 1771
Sent: 98 (retransmit: 0, fastretransmit: 0), with data: 87, total data bytes: 2275
Router-1#sh tcp vty 2

tty68, virtual tty from host aaa.eee.fff.ggg
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Local host: aaa.bbb.ddd.eee, Local port: 22
Foreign host: aaa.eee.fff.ggg, Foreign port: 53616

Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x13BBA619C):
Timer Starts Wakeups Next
Retrans 52 0 0x0
TimeWait 0 0 0x0
AckHold 51 1 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0

iss: 945887303 snduna: 945888931 sndnxt: 945888931 sndwnd: 16116
irs: 1531728475 rcvnxt: 1531729679 rcvwnd: 4008 delrcvwnd: 120

SRTT: 300 ms, RTTO: 303 ms, RTV: 3 ms, KRTT: 0 ms
minRTT: 52 ms, maxRTT: 300 ms, ACK hold: 200 ms
Flags: passive open, higher precedence, retransmission timeout

TCB is waiting for TCP Process (94)

Datagrams (max data segment is 536 bytes):
Rcvd: 91 (out of order: 0), with data: 51, total data bytes: 1203
Sent: 65 (retransmit: 0, fastretransmit: 0), with data: 61, total data bytes: 1627
Router-1#sh tcp vty 3

tty69, virtual tty from host aaa.eee.fff.ggg
Connection state is CLOSEWAIT, I/O status: 7, unread input bytes: 60
Local host: aaa.bbb.ddd.eee, Local port: 22
Foreign host: aaa.eee.fff.ggg, Foreign port: 61375

Enqueued packets for retransmit: 0, input: 3 mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x13BBA7323):
Timer Starts Wakeups Next
Retrans 59 1 0x0
TimeWait 0 0 0x0
AckHold 66 3 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0

iss: 2970881828 snduna: 2970883592 sndnxt: 2970883592 sndwnd: 16476
irs: 3850305142 rcvnxt: 3850306667 rcvwnd: 3688 delrcvwnd: 380

SRTT: 300 ms, RTTO: 303 ms, RTV: 3 ms, KRTT: 0 ms
minRTT: 72 ms, maxRTT: 300 ms, ACK hold: 200 ms
Flags: passive open, higher precedence, retransmission timeout

TCB is waiting for TCP Process (107)

Datagrams (max data segment is 536 bytes):
Rcvd: 102 (out of order: 0), with data: 67, total data bytes: 1523
Sent: 76 (retransmit: 1, fastretransmit: 0), with data: 69, total data bytes: 1763
Router-1#sh tcp vty 4

tty70, virtual tty from host aaa.eee.fff.ggg
Connection state is CLOSEWAIT, I/O status: 7, unread input bytes: 0
Local host: aaa.bbb.ddd.eee, Local port: 22
Foreign host: aaa.eee.fff.ggg, Foreign port: 21940

Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x13BBA9A54):
Timer Starts Wakeups Next
Retrans 49 0 0x0
TimeWait 0 0 0x0
AckHold 49 2 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0

iss: 1044240554 snduna: 1044241998 sndnxt: 1044241998 sndwnd: 16276
irs: 2446807786 rcvnxt: 2446808959 rcvwnd: 4048 delrcvwnd: 80

SRTT: 300 ms, RTTO: 306 ms, RTV: 6 ms, KRTT: 0 ms
minRTT: 60 ms, maxRTT: 300 ms, ACK hold: 200 ms
Flags: passive open, higher precedence, retransmission timeout

TCB is waiting for TCP Process (128)

Datagrams (max data segment is 536 bytes):
Rcvd: 195 (out of order: 0), with data: 49, total data bytes: 1171
Sent: 61 (retransmit: 0, fastretransmit: 0), with data: 55, total data bytes: 1443
Router-1#

- o - o - o - o - o - o - o - o - o -

What I noticed are the status of the connections that is

- o - o - o - o - o - o - o - o - o -

tty66, virtual tty from host aaa.eee.fff.ggg
Connection state is CLOSEWAIT, I/O status: 8, unread input bytes: 0
----------
tty67, virtual tty from host aaa.eee.fff.ggg
Connection state is ESTAB, I/O status: 1, unread input bytes: 140
----------
tty68, virtual tty from host aaa.eee.fff.ggg
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
----------
tty69, virtual tty from host aaa.eee.fff.ggg
Connection state is CLOSEWAIT, I/O status: 7, unread input bytes: 60
----------
tty69, virtual tty from host aaa.eee.fff.ggg
Connection state is CLOSEWAIT, I/O status: 7, unread input bytes: 60

- o - o - o - o - o - o - o - o - o -

and that

- o - o - o - o - o - o - o - o - o -

Rt-BorderLine-TS#who
Line User Host(s) Idle Location
66 vty 0 supergoofy UNKNOWN 5w6d ggg.hhh.iii.lll
67 vty 1 supergoofy UNKNOWN 6w0d ggg.hhh.iii.lll
68 vty 2 supergoofy UNKNOWN 5d06h ggg.hhh.iii.lll
69 vty 3 supergoofy UNKNOWN 5d05h ggg.hhh.iii.lll
70 vty 4 supergoofy UNKNOWN 5d02h ggg.hhh.iii.lll
* 71 vty 5 supergoofy idle 00:00:00 ggg.hhh.iii.lll
----------


I hope this clarify better what the situation is.
By all methods the command doesn't warn that the "deletion" wasn't applied and the session still persist to be up, or at
least in those state.
Thanks a lot for your time.

Alex.

 

For what it's worth we have 6500s which have shown a similar problem.
Here's an example from the logs:

wg4>sh user
Line User Host(s) Idle Location
1 vty 0 idle 1y4w
customer-LZC-static-224-72.cablered.com.mx
* 2 vty 1 idle 00:00:02 [a local address]

It seemed to afflict various releases of software but it hasn't come
back (perhaps we just haven't been probed) since the last upgrade.
We're currently around 12.2(18)SXF7 but the issue was present in 12.1
and (I think) earlier 12.2s as well.

In our case vty 0 is configured not to time out the exec, but the
session could not be cleared except by a reboot. I never did find a
Cisco bug report or caveat that seemed to fit, though I didn't look very
hard. There was no obvious damage to the routers.

Sam

 

Who is it? You are lucky that someone monitors your switch so long ;-)

My fears are that unclear sessions could grow. And I managed that router only by remote. I reached
the maximum number of vty logins and only due to the fact that someone on the site was asked to
connect a console cable to a computer I could enter the router. But that's a workaround and the
router is not supposed to be reached via a console cable throughout its life. I increased the number
of vty session available hoping to be always lucky.
Anyway I learned to clearly exit the session before it gets frozen or stuck.

Thanks Alex

 

Who is it? You are lucky that someone monitors your switch so long ;-)

My fears are that unclear sessions could grow. And I managed that router only by remote. I reached
the maximum number of vty logins and only due to the fact that someone on the site was asked to
connect a console cable to a computer I could enter the router. But that's a workaround and the
router is not supposed to be reached via a console cable throughout its life. I increased the number
of vty session available hoping to be always lucky.
Anyway I learned to clearly exit the session before it gets frozen or stuck.

Thanks Alex

 

A cable attached home system in Mexico? We were flattered. Not.

"sh tcp vty 0" showed only a handful of packets exchanged and a huge
number of retransmits. Looks like a bug in IOS' TCP - failing to clear
the call after retransmission failure.

I think we only ever saw them on VTYs configured with "no exec-timeout".

Were your execs configured to timeout? That's usually the default so
unless you reconfigured them or you are running an image with a
different default then this may not be the same problem.

Sam

 

A few general comments in this area:

exec-timeout affects vty sessions that are sitting at the exec prompt. If the vty
session is running an ssh/telnet client session, then the session-timeout would be
appliable instead.

TCP keepalives in and out are a good idea if you want to keep your lines clean.
Otherwise you can get clogged up with old idle TCP sessions whose peers have vanished.

There have existed some bugs where a vty can get stuck, and "clear line" is no avail.

Aaron

---

~ Sam Wilson wrote:
~
~ >
~ > I think we only ever saw them on VTYs configured with "no exec-timeout".
~
~ I think I can give further details on the topic.
~
~ I have used the router 2611 as a jump to get connected to another device.
~ Most of the time the "pending" sessions belongs to sessions that saw a starting a SSH client towards other devices.
~ So the problem could reside on it.
~ I enabled the time-out on sessions but even so I'm seeing the number of pending session increasing.
~
~ HTH Alex