And So Virtualization Comes Full Circle...

Discussion in 'NZ Computing' started by Lawrence D'Oliveiro, Oct 26, 2010.

  1. I’ve long wondered what problem virtualization was really supposed to solve.
    It was supposed to reduce the proliferation of physical servers, and thereby
    cut back on power consumption, rack space, infrastructure costs and all
    that.

    Only it turns out that’s not happening, instead by lowering the cost of
    providing extra servers, it has simply encouraged explosive growth in
    numbers of them.

    In other words, virtualization was only ever a band-aid, a solution to the
    wrong problem.

    <http://computerworld.co.nz/news.nsf/security/burning-question-how-can-vm-sprawl-be-prevented>
    Lawrence D'Oliveiro, Oct 26, 2010
    #1
    1. Advertising

  2. Lawrence D'Oliveiro

    Sweetpea Guest

    On Tue, 26 Oct 2010 23:41:48 +1300, EMB wrote:

    >> Only it turns out that’s not happening, instead by lowering the cost of
    >> providing extra servers, it has simply encouraged explosive growth in
    >> numbers of them.

    >
    > It has encouraged people like my bosses to issue an edict that "all
    > servers will be virtualised regardless of performance requirements", and
    > then jam 93 servers across 32 CPU cores. Nothing runs well, most of it
    > hardly runs at all, but it's virtual so it must be good.


    By any chance does your boss have pointy hair? ;)


    --
    "Filtering the Internet is like trying to boil the ocean"
    Sweetpea, Oct 26, 2010
    #2
    1. Advertising

  3. Lawrence D'Oliveiro

    Sweetpea Guest

    On Wed, 27 Oct 2010 18:12:10 +1300, EMB wrote:

    > On 27/10/2010 9:41 a.m., Sweetpea wrote:
    >> On Tue, 26 Oct 2010 23:41:48 +1300, EMB wrote:
    >>
    >>>> Only it turns out that’s not happening, instead by lowering the cost
    >>>> of providing extra servers, it has simply encouraged explosive growth
    >>>> in numbers of them.
    >>>
    >>> It has encouraged people like my bosses to issue an edict that "all
    >>> servers will be virtualised regardless of performance requirements",
    >>> and then jam 93 servers across 32 CPU cores. Nothing runs well, most
    >>> of it hardly runs at all, but it's virtual so it must be good.

    >>
    >> By any chance does your boss have pointy hair? ;)

    >
    > Very, very, pointy. I'm quietly getting my revenge though - this
    > evening I'm going to dump a moderately busy SQL server onto an already
    > struggling ESX host and tomorrow when the user complaints hit the
    > helldesk I'll offer a loud "told you so", some raucous laughter and
    > probably do a little dance appropriate to my heritage.


    LOL!

    Presumably you've already mentioned this as a "risk" of doing the change
    when you got that work approved?


    --
    "Filtering the Internet is like trying to boil the ocean"
    Sweetpea, Oct 27, 2010
    #3
  4. Lawrence D'Oliveiro

    Sweetpea Guest

    On Thu, 28 Oct 2010 07:17:32 +1300, EMB wrote:

    >> Presumably you've already mentioned this as a "risk" of doing the
    >> change when you got that work approved?
    >>

    > It is better than that - the change is at the request of the PHB, and he
    > is the one who drove it through the change management committee despite
    > my protests. Not my issue when blame is apportioned and questions are
    > asked :)


    Nice! :)


    > The first call about slow performance hit the helldesk at 6:47 which is
    > likely to become my issue. I can see myself moving a SQL server onto a
    > physical box again ASAP - what a pity that I'm busy for a week or so.


    LOL!

    But what if they log a high priority incident about it?

    You'll be turning back on the physical server and syncing the database on
    the physical server with the one on the virtual server.


    --
    "Filtering the Internet is like trying to boil the ocean"
    Sweetpea, Oct 27, 2010
    #4
  5. In message <ia6b9c$nol$>, EMB wrote:

    > It has encouraged people like my bosses to issue an edict that "all
    > servers will be virtualised regardless of performance requirements", and
    > then jam 93 servers across 32 CPU cores.


    But then, why did you have 93 servers in the first place? It’s always the
    Windows servers, isn’t it? Compare:

    * Why would you need 93 Windows servers? To run 93 different applications.
    * Why would you need 93 Linux servers? To get 93 times the computing power
    that a single server can give you.
    Lawrence D'Oliveiro, Oct 28, 2010
    #5
  6. Lawrence D'Oliveiro

    Sweetpea Guest

    On Fri, 29 Oct 2010 07:55:39 +1300, EMB wrote:

    >>> The first call about slow performance hit the helldesk at 6:47 which
    >>> is likely to become my issue. I can see myself moving a SQL server
    >>> onto a physical box again ASAP - what a pity that I'm busy for a week
    >>> or so.

    >>
    >> LOL!
    >>
    >> But what if they log a high priority incident about it?

    >
    > Despite a high priority incident being logged I managed to convince the
    > change management people that it would be a bad idea to do anything
    > about moving it off the ESX before we purchase new hardware for its
    > reincarnation. That gives me a week or so before I deal with it.
    >
    >> You'll be turning back on the physical server and syncing the database
    >> on the physical server with the one on the virtual server.

    >
    > I held the PHB to his edict that hardware that isn't under a support
    > contract cannot be brought into production, so I can't re-use the old
    > server.


    Nice touch - NAFfing a high priority incident (caused by management
    policy) because of management policy.

    Presumably they'll be revising their policies - or do you think they'll
    say buy more hardware for the ESX farm?


    --
    "Filtering the Internet is like trying to boil the ocean"
    Sweetpea, Oct 28, 2010
    #6
  7. Lawrence D'Oliveiro

    Sweetpea Guest

    On Fri, 29 Oct 2010 23:02:51 +1300, EMB wrote:

    >> * Why would you need 93 Windows servers? To run 93 different
    >> applications. * Why would you need 93 Linux servers? To get 93 times
    >> the computing power that a single server can give you.

    >
    > To get the segregation of data, risk and downtime that is afforded by
    > separate servers?


    Agreed. There are good practical reasons to keep applications and
    databases on separate servers.

    And it is also a good risk mitigation to have business applications/
    databases (especially key mission critical ones where non-availability
    would have a medium to high business impact) hosted on their own
    infrastructure.

    To that end from a risk management perspective it makes no sense to have
    everything virtualised and hosted on one ESX farm with one SAN (virtual
    or otherwise). Doing that gives everything at least two more single
    points of failure.

    Best practise would have 2 ESX farms physically located in different
    parts of the country, with 2 physical SANs (I'll not comment on how the
    SAN's or the ESX farms themselves should be configured as I really
    wouldn't know), with one used as DR for the other (perhaps having half of
    prod on one and half of UAT & DR on the other) with either transactional
    or log-shipping sync between prod & DR depending on how important the
    data is.

    I imagine that you'd agree with the above, but it is predicated on having
    a fairly fat pipe between them. You've probably already suggested this
    when you started down the virtualisation path, but gave you suggested
    this to your PHB using risk management/business impact corporate-speak
    language?


    Words to work into any proposal you want accepted:
    Fast, reliable, cheap, efficient.

    Words to work into any proposal you want rejected:
    Slow, unreliable, expensive, courageous.


    --
    "Filtering the Internet is like trying to boil the ocean"
    Sweetpea, Oct 29, 2010
    #7
  8. In message <iae64a$ncl$>, EMB wrote:

    > To get the segregation of data, risk and downtime that is afforded by
    > separate servers?


    You get that with one primary server and one backup server. How did you end
    up with 93?
    Lawrence D'Oliveiro, Oct 29, 2010
    #8
  9. Lawrence D'Oliveiro

    Sweetpea Guest

    On Sat, 30 Oct 2010 10:02:06 +1300, Lawrence D'Oliveiro wrote:

    >> To get the segregation of data, risk and downtime that is afforded by
    >> separate servers?

    >
    > You get that with one primary server and one backup server. How did you
    > end up with 93?


    Perhaps he is looking after machines hosting more than 93 applications or
    databases?


    --
    "Filtering the Internet is like trying to boil the ocean"
    Sweetpea, Oct 29, 2010
    #9
  10. Lawrence D'Oliveiro

    Enkidu Guest

    On 29/10/10 23:02, EMB wrote:
    > On 28/10/2010 12:41 p.m., Lawrence D'Oliveiro wrote:
    >> In message<ia6b9c$nol$>, EMB wrote:
    >>
    >>> It has encouraged people like my bosses to issue an edict that "all
    >>> servers will be virtualised regardless of performance requirements", and
    >>> then jam 93 servers across 32 CPU cores.

    >>
    >> But then, why did you have 93 servers in the first place? It’s always the
    >> Windows servers, isn’t it? Compare:

    >
    > The 93 are a mix of Windows, Linux and BSD.
    >>
    >> * Why would you need 93 Windows servers? To run 93 different
    >> applications.
    >> * Why would you need 93 Linux servers? To get 93 times the computing
    >> power
    >> that a single server can give you.

    >
    > To get the segregation of data, risk and downtime that is afforded by
    > separate servers?
    >

    And segregation of applications - you wouldn't want a JBoss and a Tomcat
    to run on the same machines. You wouldn't want to mix database and
    application server. You wouldn't want the web server or web service to
    be on the same machine (and probably not the same network segment) as
    the database and application servers. You wouldn't want to have an
    upgrade for application A to affect application B. There are many
    reasons for having multiple servers running multiple applications,
    Windows and Linux. Even before virtualization it was not common to have
    'applications' using a shared machine.

    Cheers,

    Cliff

    --

    The ends justifies the means - Niccolò di Bernardo dei Machiavelli.

    The end excuses any evil - Sophocles
    Enkidu, Oct 29, 2010
    #10
  11. Lawrence D'Oliveiro

    Sweetpea Guest

    On Sat, 30 Oct 2010 11:52:27 +1300, Enkidu wrote:

    > Even
    > before virtualization it was not common to have 'applications' using a
    > shared machine.


    Agreed. But if you're talking about applications that are not heavily
    used and that are not resource hungry then there are occasions where
    placing them on the same machine is not unreasonable. ditto for databases
    (on a shared database server).


    --
    "Filtering the Internet is like trying to boil the ocean"
    Sweetpea, Oct 30, 2010
    #11
  12. In message <iafqr1$pef$>, EMB wrote:

    > On 30/10/2010 10:02 a.m., Lawrence D'Oliveiro wrote:
    >
    >> In message<iae64a$ncl$>, EMB wrote:
    >>
    >>> To get the segregation of data, risk and downtime that is afforded by
    >>> separate servers?

    >>
    >> You get that with one primary server and one backup server. How did you
    >> end up with 93?

    >
    > You're talking about disaster recovery which is normally afforded by to
    > physically separate data centres (which we also have).


    It goes without saying the backup server is going to be in a physically
    separate location. That still doesn’t answer the question of why you have so
    many.
    Lawrence D'Oliveiro, Oct 30, 2010
    #12
  13. Lawrence D'Oliveiro

    Enkidu Guest

    On 30/10/10 14:42, Lawrence D'Oliveiro wrote:
    > In message<iafqr1$pef$>, EMB wrote:
    >
    >> On 30/10/2010 10:02 a.m., Lawrence D'Oliveiro wrote:
    >>
    >>> In message<iae64a$ncl$>, EMB wrote:
    >>>
    >>>> To get the segregation of data, risk and downtime that is afforded by
    >>>> separate servers?
    >>>
    >>> You get that with one primary server and one backup server. How did you
    >>> end up with 93?

    >>
    >> You're talking about disaster recovery which is normally afforded by to
    >> physically separate data centres (which we also have).

    >
    > It goes without saying the backup server is going to be in a physically
    > separate location. That still doesn’t answer the question of why you have so
    > many.
    >

    Some apps may require JBoss, some may need .Net. Others may use tomcat,
    some may require standalone Java. Some may require Java 1.5, some may
    only run on Java 1.6. Some web apps might only run on php4, some may
    require php5. Some might require Oracle, some might only run with MySql,
    MSSQL or PostgreSQL. Some may be NAS heads. Some may be development,
    POC, testing, UAT or production. Some may run on RH/CentOS. Some may run
    on SuSE, Debian, or BSD, or Solaris, or Tru64 or Windows of various
    flavours. Some may be virtual, others not. Some may be hot standbys or
    load balancers.

    Cheers,

    Cliff

    --

    The ends justifies the means - Niccolò di Bernardo dei Machiavelli.

    The end excuses any evil - Sophocles
    Enkidu, Oct 30, 2010
    #13
  14. Lawrence D'Oliveiro

    Enkidu Guest

    On 30/10/10 15:55, Enkidu wrote:
    > On 30/10/10 14:42, Lawrence D'Oliveiro wrote:
    >> In message<iafqr1$pef$>, EMB wrote:
    >>
    >>> On 30/10/2010 10:02 a.m., Lawrence D'Oliveiro wrote:
    >>>
    >>>> In message<iae64a$ncl$>, EMB wrote:
    >>>>
    >>>>> To get the segregation of data, risk and downtime that is afforded by
    >>>>> separate servers?
    >>>>
    >>>> You get that with one primary server and one backup server. How did you
    >>>> end up with 93?
    >>>
    >>> You're talking about disaster recovery which is normally afforded by to
    >>> physically separate data centres (which we also have).

    >>
    >> It goes without saying the backup server is going to be in a physically
    >> separate location. That still doesn’t answer the question of why you
    >> have so
    >> many.
    > >

    > Some apps may require JBoss, some may need .Net. Others may use tomcat,
    > some may require standalone Java. Some may require Java 1.5, some may
    > only run on Java 1.6. Some web apps might only run on php4, some may
    > require php5. Some might require Oracle, some might only run with MySql,
    > MSSQL or PostgreSQL. Some may be NAS heads. Some may be development,
    > POC, testing, UAT or production. Some may run on RH/CentOS. Some may run
    > on SuSE, Debian, or BSD, or Solaris, or Tru64 or Windows of various
    > flavours. Some may be virtual, others not. Some may be hot standbys or
    > load balancers.
    >

    Oh, and the security needs of the Apps may preclude server sharing,
    there may be needs of servers in different network zones, some may be
    DCs, mail servers (Exchange or other). And they are all probably tied
    together with monitors, log processors, blackberry servers and on and on
    and on. Backup servers. SAN monitors and consoles.

    Then there Citrix servers (which seem to need another server for every
    two or three new users).

    Cheers,

    Cliff


    --

    The ends justifies the means - Niccolò di Bernardo dei Machiavelli.

    The end excuses any evil - Sophocles
    Enkidu, Oct 30, 2010
    #14
  15. In message <iag7gn$6v8$>, EMB wrote:

    > On 30/10/2010 2:42 p.m., Lawrence D'Oliveiro wrote:
    >
    >> It goes without saying the backup server is going to be in a physically
    >> separate location. That still doesn’t answer the question of why you have
    >> so many.

    >
    > From your talk of "1 Linux server" I assume you would run your
    > firewall, your web sweeper, your mail front end (with AV and antispam),
    > your mail server and your webmail interface on one box Larry. Can you
    > see a slight problem with this sort of thing?


    Performance-wise Embie, no.
    Lawrence D'Oliveiro, Oct 30, 2010
    #15
  16. In message <iag65g$33a$>, EMB wrote:

    > One of the issues we have with sharing database space is the security of
    > data we handle for our clients - many of the contracts specify a
    > dedicated server.


    Would you say virtualization conforms to the letter of such an agreement,
    but not its spirit?
    Lawrence D'Oliveiro, Oct 30, 2010
    #16
  17. Lawrence D'Oliveiro

    Richard Guest

    On 30/10/2010 10:02 a.m., Lawrence D'Oliveiro wrote:
    > In message<iae64a$ncl$>, EMB wrote:
    >
    >> To get the segregation of data, risk and downtime that is afforded by
    >> separate servers?

    >
    > You get that with one primary server and one backup server. How did you end
    > up with 93?


    93 is quite extreme, but there is often the need to segregate between
    business divisions so that you are able to upgrade one without any
    impact on others, simplest way is to bill a separate server to each
    division. Otherwise you get into a who paid for what match between them.
    Richard, Oct 30, 2010
    #17
  18. Lawrence D'Oliveiro

    Sweetpea Guest

    On Sat, 30 Oct 2010 17:44:38 +1300, Lawrence D'Oliveiro wrote:

    >> One of the issues we have with sharing database space is the security
    >> of data we handle for our clients - many of the contracts specify a
    >> dedicated server.

    >
    > Would you say virtualization conforms to the letter of such an
    > agreement, but not its spirit?


    If the ESX farm is owned by that client then it conforms to the contract
    completely and in all respects.


    --
    "Filtering the Internet is like trying to boil the ocean"
    Sweetpea, Oct 30, 2010
    #18
  19. Lawrence D'Oliveiro

    Sweetpea Guest

    On Sat, 30 Oct 2010 14:42:15 +1300, Lawrence D'Oliveiro wrote:

    >>> You get that with one primary server and one backup server. How did
    >>> you end up with 93?

    >>
    >> You're talking about disaster recovery which is normally afforded by to
    >> physically separate data centres (which we also have).

    >
    > It goes without saying the backup server is going to be in a physically
    > separate location. That still doesn’t answer the question of why you
    > have so many.


    I know of one organization that has 100s of separate servers - and all
    well utilized - and many of them virtual.

    You should be asking questions about the complexity of the environment
    being supported, not bitching about the number of virtual servers.


    --
    "Filtering the Internet is like trying to boil the ocean"
    Sweetpea, Oct 30, 2010
    #19
  20. Lawrence D'Oliveiro

    Sweetpea Guest

    On Sat, 30 Oct 2010 17:43:46 +1300, Lawrence D'Oliveiro wrote:

    > In message <iag7gn$6v8$>, EMB wrote:
    >
    >> On 30/10/2010 2:42 p.m., Lawrence D'Oliveiro wrote:
    >>
    >>> It goes without saying the backup server is going to be in a
    >>> physically separate location. That still doesn’t answer the question
    >>> of why you have so many.

    >>
    >> From your talk of "1 Linux server" I assume you would run your
    >> firewall, your web sweeper, your mail front end (with AV and antispam),
    >> your mail server and your webmail interface on one box Larry. Can you
    >> see a slight problem with this sort of thing?

    >
    > Performance-wise Embie, no.


    And security wise?

    How many CPU or IO intensive applications AND databases would you
    consider to be the maximum you'd put onto one server and why?

    What would you consider to be an acceptable response time for an mission-
    critical application that is used by front line staff?

    And what value is given to availability? What would be the impact of all
    applications going unavailable if that one server dies for any reason?

    I think you're barking if you think one server only should be used for
    all applications and databases.


    --
    "Filtering the Internet is like trying to boil the ocean"
    Sweetpea, Oct 30, 2010
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Col's Cavern

    Jisatsu circle (Suicide Circle)

    Col's Cavern, Jun 7, 2004, in forum: DVD Video
    Replies:
    1
    Views:
    565
    Justin
    Jun 7, 2004
  2. Veronica
    Replies:
    14
    Views:
    649
    Veronica
    Aug 27, 2005
  3. Mark Gillespie

    x64 and Vanderpool virtualization.

    Mark Gillespie, Jul 18, 2006, in forum: Windows 64bit
    Replies:
    3
    Views:
    507
  4. =?ISO-8859-1?Q?R=F4g=EAr?=

    Full Circle

    =?ISO-8859-1?Q?R=F4g=EAr?=, May 17, 2007, in forum: Computer Support
    Replies:
    3
    Views:
    506
    =?ISO-8859-1?Q?R=F4g=EAr?=
    May 18, 2007
  5. - Bobb -
    Replies:
    0
    Views:
    506
    - Bobb -
    May 12, 2010
Loading...

Share This Page