Lotus: Lotus / Domino Government Solutions 3 - A Variety of Examples 2000.

THEMES: Lotus\e-Government
YEAR: 2000
Login Login
User: Anonymous

LABEL: e-Government
PEOPLE: Buschor, Ernst
THINGS: Kickoff | Public Administration | Verwaltung | Workshop
TIME: 2000 | 2001
State of Georgia Pardons and Paroles Board
    State of Georgia Pardons and Paroles Board
    Balancing Cost and Performance: Compupro and the Georgia Pardons and Paroles Board Tune a High Performance Solution

    Informationsmanagement Begnadigungsverfahren


    Until early in 1998, parole officers in 70 offices through the State of Georgia were relying on a manual system for tracking parolees. That was before the Georgia Pardons and Paroles Board, working with Lotus Business Partner
    Compupro, started to roll out an automated Case Management System, powered by the Lotus Domino server and Notes client.

    The system is designed to enable parole officers to work remotely from the field.
    Like so many organizations, change has not been gradual. They are rolling out Notes at the same time that they are both building and deploying out the Case Management System. The number of users is growing while the system is in service and data is live. By May 1998, 600 users, the majority of whom were scattered throughout the State, were using the system. Users working locally were entering information into the system to record their interactions with each of their assigned cases. Mornings and late afternoons, when virtually all of the remote users replicated their changes, there were performance problems.

    The performance problems were not unexpected. Compupro President, Jim Carzoli says, "Everything had gone according to plan. We did expect to have a performance problem because of the size of the database and the size of the views. We knew we'd be pushing the limit on a lot of this stuff."

    With so many users working simultaneously and the enormous database constantly indexing and re-indexing, performance slowed to where it was taking minutes, rather than seconds to open views. Zach Mabe, Principal at Compupro who designed and is helping to deploy Case Management, says "This is the single biggest Notes database I've ever seen. We designed it to be super simple for the user. We have one main view that contains all the basic information because the users didn't want to have to go clicking everywhere." But that simplicity for the user has come at a cost -- large views require processor-intensive indexing nearly constantly.


    Problem solving strategy

    Compupro started to develop strategies for how to improve performance. They needed to prioritize less costly alternatives before big ticket items. David Sheffield, Georgia Pardons and Paroles Board Enterprise Network Manager, explains. "We won't have money for a number of these things until after July. So we're working hard to make sure that we're ready to implement the best alternative as soon as the money comes through."

    Here are some of the potential solutions they came up with:
  • Re-design database -- By splitting out views, dividing views up into the six or seven regions, they could reduce view size by about a quarter or more.
  • Clustering -- By clustering the program on two machines and devoting one of the machines to Case Management only, setting a high threshold for failover, they felt they could reduce the number of people sitting on the server with open sessions.
  • Adding processors -- With additional processors, they felt they could take away some of the indexing load by starting another indexer task, putting less pressure on the server to deliver information. The indexes would also be kept up to date better.
  • Upgrade to a more powerful server -- Long term, they know the application is going to get bigger and will eventually need to connect to an ORACLE database; a move to a larger, more powerful server is a long term possibility. - -

    Analyzing network traffic

    Before implementing any of these potential solutions, Compupro wanted to gather more information and eliminate any network problems.

    They brought in a sniffer, an electronic device that analyzes the network and network traffic on the network backbone, routers, switches and IP packets. They were surprised by the results. "The results indicated excessive re-transmissions of packets on the network and excessive physical errors," says Carzoli. However, he still felt this would not cause the performance degradation they were experiencing.

    - -

    Implementing solutions

    They cleaned up the database by deleting some views that were no longer needed and got somewhat better performance.

    They tried to eliminate the errors by replacing the NIC card, but continued to have similar errors.
    They hypothesized that problems might be caused by the bus architecture of the server itself. So they switched everything to another server, and although performance somewhat improved, there were still errors.

    Finally, they pulled out their NT manual and went through the sections on performance analysis on the disk subsystem. "As we went through all the analyses on the NT Performance Monitor, we came across a counter of the number of pages per second and an acceptable range of zero to 20. We were finding over 1700 pages per second," says Compupro President Jim Carzoli.

    They knew they were onto something, but first wanted to be sure the measurement they were getting was correct. So they re-ran the analyses and confirmed the results.

    The solution, according to the manual, was to add RAM. They computed how much was needed -- another gigabyte.

    Success! Suddenly, even at peak times, response time dropped to seconds. When they re-tested, they found pages per second had dropped from 1700 to approximately 40. Carzoli summarizes, "Once we added that gig of RAM, performance was excellent. The customer is completely satisfied."

    - -

    Lessons Learned

    Carzoli sums up the lessons learned: "We had already learned not to jump to conclusions. Adding more processors would have been a costly mistake and would not have alleviated the problem. However, we did learn that if we see or hear the hard disks churning away, it is a good indication that more RAM is needed to reduce the constant paging of the disk array."
    - -

    About Compupro

    Compupro, a Lotus Premium Partner and IBM Best Team member, is an Atlanta based consulting firm founded in 1981. Compupro brings leadership, management and Domino and Notes expertise to their client engagements. They design, develop and implement leading-edge technical solutions to solve their clients' business problems.

    System Basics

    An NT server with two 200-Megahertz Pentium Pro chips
    256 MG of RAM
    25 GB of storage
    NT4.0 and Domino 4.5.1
    56 KB frame relay connection for remote offices
    10 MB network
    1.5 GB Database with 400,000 documents with 40+ views