Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > NTLM authentication with httpclient

Reply
Thread Tools

NTLM authentication with httpclient

 
 
Jim Clark
Guest
Posts: n/a
 
      12-01-2007
I have rewritten my net/http script that I had questions on a couple of
days ago using httpclient but am stuck on the NTLM authentication piece.
This is what I have so far:

#!c:/ruby/bin/ruby.exe
CERT_FILE = "c:/certs/jim_nopw2.pem"
CA_CERT = "c:/ca_certs/servers_ca.cer"

require 'rubygems'
require 'httpclient' # using v2.1.2 and rubyntlm 0.1.1 installed

client = HTTPClient.new
client.ssl_config.set_trust_ca(CA_CERT)
client.ssl_config.set_client_cert_file(CERT_FILE, CERT_FILE)

# This fetches the page but 401 error because not authenticated through NTLM
resp = client.get("https://some_website.com/default1.asp")

# Not sure how to turn on NTLM authentication and feed arguments. I
tried this but still get 401 error
#resp = client.request('GET', "https://some_website.com/default1.asp",
{'username' => 'DOMAIN\username', 'password' => 'password'})
puts resp.content
puts resp.status

Thanks again for any and all help.

-Jim

 
Reply With Quote
 
 
 
 
yermej
Guest
Posts: n/a
 
      12-01-2007
On Dec 1, 1:02 am, Jim Clark <(E-Mail Removed)> wrote:
> I have rewritten my net/http script that I had questions on a couple of
> days ago using httpclient but am stuck on the NTLM authentication piece.
> This is what I have so far:
>
> #!c:/ruby/bin/ruby.exe
> CERT_FILE = "c:/certs/jim_nopw2.pem"
> CA_CERT = "c:/ca_certs/servers_ca.cer"
>
> require 'rubygems'
> require 'httpclient' # using v2.1.2 and rubyntlm 0.1.1 installed
>
> client = HTTPClient.new
> client.ssl_config.set_trust_ca(CA_CERT)
> client.ssl_config.set_client_cert_file(CERT_FILE, CERT_FILE)
>
> # This fetches the page but 401 error because not authenticated through NTLM
> resp = client.get("https://some_website.com/default1.asp")
>
> # Not sure how to turn on NTLM authentication and feed arguments. I
> tried this but still get 401 error
> #resp = client.request('GET', "https://some_website.com/default1.asp",
> {'username' => 'DOMAIN\username', 'password' => 'password'})
> puts resp.content
> puts resp.status
>
> Thanks again for any and all help.
>
> -Jim


There are examples included in the rubyntlm gem. I'm running Windows
so they're in C:\ruby\lib\ruby\gems\1.8\gems\rubyntlm-0.1.1\examples\.
YMMV.
 
Reply With Quote
 
 
 
 
Jim Clark
Guest
Posts: n/a
 
      12-01-2007
yermej wrote:
>> There are examples included in the rubyntlm gem. I'm running Windows
>> so they're in C:\ruby\lib\ruby\gems\1.8\gems\rubyntlm-0.1.1\examples\.
>> YMMV.
>>

I'm really trying to do this in httpclient versus getting down into
rubyntlm which shouldn't be necessary. I took another look at the
httpclient.rb source and found what I was looking for on the get method
which allows an extheader hash to be passed with arguments. Now I feel I
am real close but I am still getting a 401 error. The script is now
looking like:

#!c:/ruby/bin/ruby.exe
CERT_FILE = "c:/certs/jim_nopw2.pem"
CA_CERT = "c:/ca_certs/servers_ca.cer"

require 'rubygems'
require 'httpclient' # using v2.1.2 and rubyntlm 0.1.1 installed

client = HTTPClient.new
client.ssl_config.set_trust_ca(CA_CERT)
client.ssl_config.set_client_cert_file(CERT_FILE, CERT_FILE)

resp = client.get("https://some_website.com/default1.asp"), nil,
{'username' => 'DOMAIN\username', 'password' => 'password'})
puts resp.content
puts resp.status

When I look at resp.content in my debugger, I see that the @header_item
array has the normal info (content type, content length, server, etc.)
and also has ["WWW-Authenticate", "Negotiate"] and the second
["WWW-Authenticate", "NTLM"]. When I trace through the get method, I see
that @negotiate_auth has a hash @auth which == {} and @auth_default
which == null. I guess if I can figure out how to populate these
correctly then I'm probably good to go.

This will have to wait until tomorrow or Sunday night and a few more
drinks. However, I'll send another S.O.S. that if someone has already
figured this out and can show me a good code snippet, I'd be more than
willing to share a few drinks (if you are around Seattle) or offer high
praise and many thanks from afar.

-Jim

 
Reply With Quote
 
Francis Cianfrocca
Guest
Posts: n/a
 
      12-01-2007
Note: parts of this message were removed by the gateway to make it a legal Usenet post.

On Dec 1, 2007 4:08 AM, Jim Clark <(E-Mail Removed)> wrote:

>
> When I look at resp.content in my debugger, I see that the @header_item
> array has the normal info (content type, content length, server, etc.)
> and also has ["WWW-Authenticate", "Negotiate"] and the second
> ["WWW-Authenticate", "NTLM"]. When I trace through the get method, I see
> that @negotiate_auth has a hash @auth which == {} and @auth_default
> which == null. I guess if I can figure out how to populate these
> correctly then I'm probably good to go.
>



Keep in mind that NTLM requires at least two round trips to the server in
each connection. Your first GET/POST/whatever request needs to have an NTLM
"type 1" message in the Authorize header. The server will then respond with
a 401, but the response will contain an NTLM "type 2" message that you use
to create an NTLM "type 3" response. You then send your GET/POST again, with
the type 3 response in the Authorize header. At that point, if all goes
well, you get a 2xx from the server.

Your best bet is to use rubyntlm as a previous commenter suggested.

 
Reply With Quote
 
Phrogz
Guest
Posts: n/a
 
      12-01-2007
On Dec 1, 12:02 am, Jim Clark <(E-Mail Removed)> wrote:
> I have rewritten my net/http script that I had questions on a couple of
> days ago using httpclient but am stuck on the NTLM authentication piece.
> This is what I have so far:


FWIW, I've worked around this in the past by using a download of curl
that supports ntlm, and wrapped that in a shell command.

Note that if you want binary data (which it doesn't sound like you do)
and are on windows, you'll need to go through a temporary file to work
around this bug:
http://rubyforge.org/tracker/?func=d...=426&atid=1698
 
Reply With Quote
 
Jim Clark
Guest
Posts: n/a
 
      12-02-2007
Francis Cianfrocca wrote:
> Keep in mind that NTLM requires at least two round trips to the server in
> each connection. Your first GET/POST/whatever request needs to have an NTLM
> "type 1" message in the Authorize header. The server will then respond with
> a 401, but the response will contain an NTLM "type 2" message that you use
> to create an NTLM "type 3" response. You then send your GET/POST again, with
> the type 3 response in the Authorize header. At that point, if all goes
> well, you get a 2xx from the server.
>
> Your best bet is to use rubyntlm as a previous commenter suggested

I admit that I'm a bit stubborn sometimes. If my wife subscribed to this
list, I'm sure she would provide ample evidence.

Since the httpclient2 2.1.2 release (2007-09-22) specifically says it
supports "NTLM auth for WWW-Authenticate", I'm reluctant to give up on
it. When I look at the httpclient.rb source, the request method may loop
up to 5 times depending and the NegotiateAuth class has all the logic in
it to handle the NTLM. Tracing through, the @www_auth.set_auth method
never seems to be called at least using the client.get call. I tried to
force the set_auth call like this:

def create_request(method, uri, query, body, extheader, proxy)
if extheader.is_a?(Hash)
extheader = extheader.to_a
end
if cookies = @cookie_manager.find(uri)
extheader << ['Cookie', cookies]
end
boundary = nil
content_type = extheader.find { |key, value|
key.downcase == 'content-type'
}
if content_type && content_type[1] =~ /boundary=(.+)\z/
boundary = $1
end
req = HTTP::Message.new_request(method, uri, query, body, proxy,
boundary)
+ myuser, mypassword = nil, nil
extheader.each do |key, value|
req.header.set(key, value)
+ myuser = value if key == "username"
+ mypassword = value if key == "password"
end
+ @www_auth.set_auth(uri, myuser, mypassword) if !myuser.nil? &&
!mypassword.nil?
if content_type.nil? and !body.nil?
req.header.set('content-type', 'application/x-www-form-urlencoded')
end
req
end

And another change in line 1703 to allow access to @www_auth,
- attr_reader :www_auth
+ attt_accessor :www_auth

Sorry for not providing a real diff file. I don't have it on my windows
machine and I don't feel like fetching it at this late hour.

Point is that @www_auth.set_auth is now called within each of the
do_get_block calls on line 1956 of the request method but I am still
getting a 401.1 not authorized error.

I'm about done on following up on this though. In working through the
certificate problems for my httpclient script yesterday, I tried the
same changes in my Perl LWP script tonight and it is now working with
SSL, client certificates and NTLM authentication. Since I need a working
script, I'm moving forward now with my Perl script and dropping any more
efforts on the Ruby version. I realize I may get a few boo/hisses by
saying that on a Ruby list but I've already lost too many hours sleep
already!

If Hiroshi or anyone else gets NTLM working, please provide an example
script to show how it's done. I'd love to use this library because I
like what I see in httpclient.

Regards,
Jim

 
Reply With Quote
 
yermej
Guest
Posts: n/a
 
      12-02-2007
On Dec 2, 5:07 am, Jim Clark <(E-Mail Removed)> wrote:

> If Hiroshi or anyone else gets NTLM working, please provide an example
> script to show how it's done. I'd love to use this library because I
> like what I see in httpclient.
>
> Regards,
> Jim


I don't have an NTLM server to test against at this point, but from
looking at the code you might want to try (with the original
httpclient.rb) calling set_auth on the client rather than passing in
an extra header, thusly:

CERT_FILE = "c:/certs/jim_nopw2.pem"
CA_CERT = "c:/ca_certs/servers_ca.cer"

require 'rubygems'
require 'httpclient' # using v2.1.2 and rubyntlm 0.1.1 installed

client = HTTPClient.new
client.ssl_config.set_trust_ca(CA_CERT)
client.ssl_config.set_client_cert_file(CERT_FILE, CERT_FILE)

client.set_auth(nil, 'DOMAIN\username', 'password')

resp = client.get("https://some_website.com/default1.asp")
puts resp.content
puts resp.status

I think I'll be able to play with this once I'm back at work tomorrow.
I'll post more if I can get it working. As you said, all the pieces
are there, it's just a matter of figuring out how to use them
correctly.
 
Reply With Quote
 
yermej
Guest
Posts: n/a
 
      12-03-2007
On Dec 2, 3:11 pm, yermej <(E-Mail Removed)> wrote:
> On Dec 2, 5:07 am, Jim Clark <(E-Mail Removed)> wrote:
>
> > If Hiroshi or anyone else getsNTLMworking, please provide an example
> > script to show how it's done. I'd love to use this library because I
> > like what I see in httpclient.

>
> > Regards,
> > Jim

>
> I don't have anNTLMserver to test against at this point, but from
> looking at the code you might want to try (with the original
> httpclient.rb) calling set_auth on the client rather than passing in
> an extra header, thusly:
>
> CERT_FILE = "c:/certs/jim_nopw2.pem"
> CA_CERT = "c:/ca_certs/servers_ca.cer"
>
> require 'rubygems'
> require 'httpclient' # using v2.1.2 and rubyntlm 0.1.1 installed
>
> client = HTTPClient.new
> client.ssl_config.set_trust_ca(CA_CERT)
> client.ssl_config.set_client_cert_file(CERT_FILE, CERT_FILE)
>
> client.set_auth(nil, 'DOMAIN\username', 'password')
>
> resp = client.get("https://some_website.com/default1.asp")
> puts resp.content
> puts resp.status
>
> I think I'll be able to play with this once I'm back at work tomorrow.
> I'll post more if I can get it working. As you said, all the pieces
> are there, it's just a matter of figuring out how to use them
> correctly.


I've played with this some and there seem to be a couple problems. Or
I'm just doing things completely wrong.

It seems that calling set_auth with the first parameter as nil should
end up setting the @auth_default instance variable in the
NegotiateAuth object. It does not. So, using the target URL as the
first parameter is the first step to getting things almost working.

Once I did that, I was getting to the response stage of the
authentication, but it wasn't successful. At this point, that seems
like it might be a problem with the way HTTPClient uses rubyntlm or it
might not support the version our server is using or...

It's a bit soon to say, but it seems that there might be a bug in
HTTPClient. I'll post again once I've figured it out.
 
Reply With Quote
 
yermej
Guest
Posts: n/a
 
      12-03-2007
It works now! I just made a stupid mistake in my testing. This is an
example that should work:

> > CERT_FILE = "c:/certs/jim_nopw2.pem"
> > CA_CERT = "c:/ca_certs/servers_ca.cer"

>
> > require 'rubygems'
> > require 'httpclient' # using v2.1.2 and rubyntlm 0.1.1 installed

>
> > client = HTTPClient.new
> > client.ssl_config.set_trust_ca(CA_CERT)
> > client.ssl_config.set_client_cert_file(CERT_FILE, CERT_FILE)

>
> > client.set_auth("https://some_website.com/default1.asp", 'DOMAIN\username', 'password')

>
> > resp = client.get("https://some_website.com/default1.asp")
> > puts resp.content
> > puts resp.status


I forgot to include the domain in my previous attempts as it isn't
required in the browser. I'm guessing the browser grabs the target/
domain during the ntlm process and prepends it to the username. It
wouldn't be hard to modify httpclient to do the same, but isn't
completely necessary.

I think it should also work to call set_auth with nil as the first
parameter and have that set the default credentials, but that doesn't
work the way httpclient is currently coded. NegotiateAuth#set seems to
allow for it though. I hope this helps and will allow you to continue
on your way with Ruby.
 
Reply With Quote
 
Jim Clark
Guest
Posts: n/a
 
      12-03-2007
yermej wrote:
> It works now! I just made a stupid mistake in my testing. This is an
> example that should work

Very cool and thank you!

Regards,
Jim

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
ANN: python-ntlm - provides NTLM support, including an authenticationhandler for urllib2 Matthijs Python 0 12-10-2008 03:38 PM
Java - Integrated Windows Authentication - NTLM Authentication Forwarding Will Java 5 12-03-2005 01:00 AM
httpclient authentication ravi Java 1 04-13-2005 10:45 AM
Silent NTLM Autentification from httpclient Pavel Java 2 08-29-2004 05:12 PM
WebControls and NTLM Authentication Carlos Fersura ASP .Net 0 11-03-2003 04:48 PM



Advertisments