scatter/gather test

Discussion in 'Linux Networking' started by wkevin, Jul 17, 2012.

  1. wkevin

    wkevin Guest

    I want to perform a test which uses scatter/gather.
    This is to say, I set scatter/gather via ethtool
    with ethtool -K eth0 sg on

    scatter gather should work when nr_frags > 0

    Many drivers have special treatment for scatter/gather

    see for example : drivers/net/ethernet/realtek/8139cp.c
    and the cp_start_xmit() method.

    Is there a way by user space tools or by some program
    to generate tx traffic on the nic so that
    skb->shinfo(nr_frags) will be greater than 0 in the tx path?

    wkevin, Jul 17, 2012
    1. Advertisements

  2. wkevin

    Rick Jones Guest

    Using sendfile() might do that, as might something like a netperf
    TCP_STREAM test with a smallish send size. You would want to
    instrument the driver in question to verify you were indeed getting
    fragments though.

    rick jones
    Rick Jones, Jul 17, 2012
    1. Advertisements

  3. wkevin

    wkevin Guest

    Thanks for you answer
    I tried with netperf like this:
    netperf -H -l 200
    and also:
    netperf -H -l 6000

    I added print of skb_shinfo(skb)->nr_frags in the tx path of the driver
    In both cases scatter/gather is not performed. And I see skb_shinfo(skb)->nr_frags=0, and not greater than 0 (as should be in the
    case of scatter/gather).

    ethtool -k eth0 shows that sg is on.

    Any ideas ? did anybody had success with testing scatter/gather?
    wkevin, Jul 18, 2012
  4. wkevin

    Noob Guest

    Have you tried coalescing multiple disjoint buffers using sendmsg?
    (sendmsg is the text-book API for user-space scatter/gather.)

    ssize_t sendmsg(int socket, const struct msghdr *message, int flags);

    The msghdr structure shall include at least the following members:

    void *msg_name Optional address.
    socklen_t msg_namelen Size of address.
    struct iovec *msg_iov Scatter/gather array.
    int msg_iovlen Members in msg_iov.
    void *msg_control Ancillary data; see below.
    socklen_t msg_controllen Ancillary data buffer len.
    int msg_flags Flags on received message.

    Noob, Jul 18, 2012
  5. wkevin

    wkevin Guest

    wkevin, Jul 18, 2012
  6. wkevin

    wkevin Guest

    Thanks for your answer. I started
    coding according to the sendmsg links.

    Three more questions that might help me, if I may:

    Should msg_iovlen for scatter gather case be greater than 1 ? this is what I think.
    and should msg_controllen be greater than 1 ? or msg_control can be null ?

    Will scatter gather work with this sendmsg API with UDP/TCP/RAW sockets , or in only one or two of them ?

    wkevin, Jul 18, 2012
  7. wkevin

    Noob Guest

    [ Please, don't top-post ]
    Right. sendmsg is supposed to coalesce ("gather") the data scattered
    in the msg_iov[k].iov_base buffers.

    POSIX states:
    If you don't want to deal with all the complexity of sendmsg,
    you can also use writev for connected sockets.
    I believe it's supposed to work with all types of sockets.

    Noob, Jul 18, 2012
  8. wkevin

    Jorgen Grahn Guest

    Read the sendmsg() manual page! Mine says quite clearly "elements in
    msg_iov". The "elements" are the iovec structs, and each of them
    contains a chunk of octets.

    If you want to be overwhelmed by information, you can also install the
    POSIX manual pages. sendmsg(3p) is even more detailed, but also harder
    to read.
    Read the sendmsg() manual page! Or try it yourself.
    "Ancillary data" is an obscure feature which you can happily ignore.
    Read the sendmsg() manual page! Or try it yourself. But no such error
    code is listed, so it should work everywhere, unless documented
    otherwise in udp(7), tcp(7), raw(7) and so on. (Of course no hardware
    NIC support will be involved when you use it for Unix domain sockets
    et cetera.)

    Jorgen Grahn, Jul 18, 2012
  9. wkevin

    Rick Jones Guest

    All that does is change the runtime of the test. 200 seconds in the
    first case, and 6000 seconds in the second.

    If you want to use sendfile() via netperf that would be the
    "TCP_SENDFILE" test:

    netperf -t TCP_SENDFILE ...

    If you want to change the size of what netperf puts into the socket
    buffer each time, that would be a "test-specific" (after the "--" on
    the command line) -m option:

    netperf <global options> -- -m <size>

    for example:

    netperf -H -- -m 128

    will cause netperf to call send() with 128 bytes each time.

    rick jones
    Rick Jones, Jul 18, 2012
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.