A rchive Date
[ 10-06-2000 ]
Category
[ Information Technologies ]
sub-Categoy
[ Lotus ]
|
[Domino Performance
What are the criteria you're using to evaluate server performance for Domino R5 and beyond?
Greater message throughput, higher server availability, bigger databases and directories, faster agent execution, and yes, the ability to support more concurrent users than ever before. These are just some of the areas where the Performance Team is continuing to tune the Domino server.
James Grigsby
We're primarily focused on the widespread desire among customers to consolidate the number of servers that they use so they can reduce the cost of their networks. We're also looking for ways to increase server availability to improve end user access - this is especially important as companies' Web sites become more critical to their business and these sites need to be constantly available. And, we're looking at leveraging some of the newer systems' capabilities, like being able to exploit faster processors, take advantage of more processors and more memory if it's available, and being able to exploit bigger and faster systems like the AS/400, OS390, and high-end UNIX systems like Solaris (Sparc), RS/6000, and HP-UX. With Domino R5, we have a much larger database file size capability, which leverages the larger file systems that these platforms can handle. At the low level, we're focusing on semaphores, cache hit rates, I/O queues, CPU utilization, memory utilization, and network utilization at a high load. We're learning a lot and providing feedback to kernel developers and hardware vendors so they can enhance or optimize their configurations.
What are your performance goals for Domino?
James Grigsby
We have a lot of goals, but just to give you an idea of what we're focusing on, we're looking to improve by a factor of two to five times, the number of users supported by a single server. We also aim at improving by two to ten times, directory size, message throughput, and agent execution. One of the main ways we're accomplishing these goals is by better supporting the Symmetric MultiProcessing (SMP) architecture for better scalability. The performance work is concurrent and an iterative process with R5.x development. This means important cross-platform performance decisions are made early in the feature design process. The result is a more reliable server load for customers across the various platforms.
What are the most noticeable changes for Domino R5 administrators?
James Grigsby
Administrators are going to see very noticeable improvements in the number of concurrent users that they can support, message throughput, and more efficient use of server assets. Also, they'll be able to better exploit additional server resources, like more memory, CPUs, and I/O subsystems with newer server configurations.
How has the Web affected your approach to performance tuning and capacity planning?
George Demetriou
Typically, models of performance follow models of usage. With the Web industry, everything is happening concurrently. For instance, some users are retrieving data from the file system, or data can come from Domino databases, as well as back-end relational database management systems, Web-based simple messaging, or messaging systems with calendar and scheduling and workflow applications. This is giving us a chance to define new models for performance tuning and capacity planning because you have such different types of Web "users" based on whatever the user happens to be doing.
How has your approach changed?
James Grigsby
The number of Web-centric usage profiles has increased. Our first efforts included WebWalker, which involves the HTTP read-only capabilities of a single .NSF file, and WebBuyer, which provides HTTP read-write of a single .NSF. The initial published performance data shows a configuration that supports 200 concurrent users. You can see that report on the NotesBench Consortium Web site. Now, we're expanding our efforts with R5 performance to include messaging, calendar and scheduling, and authentication. This models many users concurrently accessing dynamically generated and newly delivered content, such as messages and meetings.
What are some of the major things you would recommend people do to improve server performance?
Carol Zimmet
To see a list of 10 things we recommend you look at to improve server performance, see the "Iris Today" sidebar, "The top 10 ways to improve your server performance."
In general, people should be making use of their platform performance tools in addition to their Notes-specific performance information. This helps them to better understand the information returned and relate their performance to the published benchmarks.
Also, there are certain key areas to pay attention to - for example, we've been recently observing that disk I/O is the bottleneck for many production environments. Specific statistics to look at are Average Disk Queue Length, Average Disk Sec/Read or Write, and paging. We're publishing suggestions about these key areas on the Notes.net site, as well as incorporating suggestions into the documentation.
How do you accurately gauge performance comparisons with competitive products when most Web servers are serving static content?
James Grigsby
That's a challenge. We focus on dynamic content, such as integrated business logic, or newly arrived mail, which is constantly changing. This is how our customers are using Domino, so this is where we focus performance tuning. Other vendors will have to move there as well to attain the scalability and performance required for constantly changing data and business workflows.
What kinds of tools for capacity planning are included with Domino R5?
Carol Zimmet
You can download Domino Server.Planner from Notes.net. This is a Notes application that we released with Domino R4.6 and enhanced for R4.61. We're also revamping it for R4.63. It's designed to query production environment loads against certified vendor data across a range of systems. Hardware vendors could not produce data for each load point, so we're enhancing Server.Planner to do extrapolation between data points. This means if the published data includes 750 and 1200 user datapoints, you can also obtain a 950 user extrapolation without an additional lab run. You can read more about Server.Planner in the "Iris Today" article "Simulating your Load Environment with Domino.Server.Planner."
There is also Server.Load, which provides an easy-to-use GUI to generate a load against a Domino server. It allows you to easily monitor Notes statistics and metrics under a simulated load. Server.Load is available in the Lotus Performance Zone.
What kinds of benchmarking tools are you using?
Rich Kanosky
We use both internal tools as well as public tools, such as NotesBench and Server.Load, to isolate bottlenecks on the back-end or Domino server. The public tools allow real-world workloads (like R5Mail, R5IMAP, WebMail, SMTP/POP3, and so on) to be run against the Domino server. By using these workloads, we are capable of determining the capacity and response times for a simulated number of users running a particular workload on the Domino server.
In addition to the above tools, we are also using OS-specific benchmark utilities, such as AIX Performance ToolBox and Windows NT Performance Monitor, to provide additional insight and assist in pinpointing bottlenecks (like CPU utilization, disk I/O, memory statistics, and so on) on the server's back-end.
What about the Notes Router - what kind of performance improvements can we expect?
Brian Richards
We're looking for improvements in both local delivery and message transfer for native Notes and SMTP traffic. The improvements are based on code improvements, database refinement in MAIL.BOX attributes, and the ability to have multiple MAIL.BOX files on a server, as well as multiple threads for local delivery of messages. We are able to deliver tens of thousands of messages daily with R4. We want to move this to hundreds of thousands of messages, without a backlog. This is one example of the convergence of performance measured in terms of the load, with availability measured by the reliable and timely delivery of messages.
What can you tell us about performance of the different Internet protocols, such as IMAP?
James Grigsby
We're working aggressively on these. In general, the user/server metric will double. The protocols are better able to leverage other Domino server capabilities, such as the new data store and server framework. Because the protocols are more tightly integrated and able to leverage other server infrastructure code, we are already seeing capacity and performance improvements.
What about POP3 performance vs. Exchange?
James Grigsby
While our latest benchmark results are in the 2000+ range for a departmental class server (without leveraging the flexibility of moving the MTA to a different server which enhances the number even more), we feel we're competitive with Exchange on NT now. We also have a customer deployment. The Bankinter bank, featured on the Lotus Performance Zone, uses a single POP3 domain on a single Domino server. POP3 allows users to connect, retrieve and send mail, and then disconnect. Thus, the number of users connected to the service at a single time is not high, but the bank can provide this service from one Domino server to over 55,000 users, including supporting names for each of them. There are over 30,000 messages on peak days. A significant advantage is that the maintenance of user accounts (that is, file changes) doesn't impact other users since each customer has their own mail file. This user/file granularity allows administrators to avoid the risk of a single Domino message file corruption impacting other users.
On Exchange, the message store file corruption can cause denial of messaging service to all server users. Domino message store flexibility allows administrators to overcome the most common hardware bottleneck, I/O, by distributing Domino databases across multiple I/O buses, controller channels, and disks to leverage the server hardware's capability. Exchange's single file architecture compounds I/O contention and increases the denial-of-service risk to all users when a message store corruption occurs. Even their clustering solution doesn't help with this type of storage corruption because it's not file access, but file content (message store data) that's the problem.
How are you measuring search performance - what kinds of improvements can we expect?
James Grigsby
We're still working on this, but the bottom line is that you can expect improvements because of the new ODS (On Disk Structure), which is designed to make indexing and searching better.
What kind of performance should we expect as user loads increase?
Mike Robinson
Our goal is to increase performance two to five times, without using partitioning, which will allow us to add even more users. Partitioning of Domino instances, Domino tasks, and features within a single server and across multiple servers keeps Domino capacity from being limited by current operating system and hardware limits.
How does Domino R5 perform across various platforms?
Harry Murray
We are seeing significant performance improvements across all platforms. The coding of the UNIX versions of R5 takes advantage of some of the specific UNIX features that help performance. We are seeing that the UNIX versions are equal or better in performance than the NT versions. If you run Domino partitioned servers on UNIX, you should be able to support even higher loads, since 64-bit UNIX enterprise systems can address more memory (greater than 4GB) and leverage large file storage space. Domino is able to take full advantage of large, stable UNIX enterprise systems.
What about Sun platforms, in particular?
Nirmala Venkatraman
Sun machines have already proven their CPU scalability in supporting a high number of concurrent users with Domino server partitions. In R5, we are seeing Domino on Solaris scaling much better in terms of capacity, throughput, and response times, with a single server instance. This is possible due to the core Domino architectural changes and UNIX -specific performance improvements. With implementations of asynchronous I/O being worked on, we should see a reduction in system resource utilization by the Domino server, and hence a better overall scalability. Solaris 2.6 supports large file systems - larger than 2GB - which should help Domino R5 large database support, which is in excess of 4GB. We are already seeing good performance improvements on the IMAP server on Solaris going from R4.6x to R5. With more performance optimizations being done on the IMAP server, we should see a tremendous performance gain of the IMAP server across all platforms.
What about the AS/400 - is that going to represent a real performance win for you?
Harry Murray
Yes, because of its support for 64-bit hardware, operating systems, and applications. There is a native version of Domino for the AS/400, which provides much faster messaging, database, and Web applications.
How do you stack up against Exchange 5.5 in general?
James Grigsby
We're really working on reliability at load. What this means is that as our load increases, databases aren't corrupted, and in the event of an error, the "penalty" is transparent to the administrators and users. For example, because the server can repair the database online while still under load, messages are delivered and available to the user. Load-balancing clusters, and the ability for online backups done from a replica, are other capabilities that make errors transparent. This means the "risk" of consolidating more users onto a single server is reduced.
Another advantage is that we're leveraging the standards and capabilities of the other platforms that Exchange doesn't support. Plus, we're expecting to be on par with Exchange on NT from a performance standpoint, but we'll be more reliable because with multiple mail databases, we don't have the single point of failure that their shared mail store has. Corruption in, or restoration of one user's account doesn't have an impact on other users. We already have active/active server cluster members, which improves Domino server availability. By not being limited to two servers, a cluster can be up constantly, even when one member is brought down for upgrades to Domino R5 or hardware.
We can also perform beyond limits imposed by NT because we support other enterprise platforms, such as Solaris, AIX, AS/400, HP-UX, and OS-390. These platforms measure availability in months-to-years of scheduled uptime.
What about monitoring tools - anything new here?
Carol Zimmet
We're working on tools to monitor real-time performance analysis and trend analysis, which began appearing in Domino R5 and will incrementally improve with future releases. We're also working on methods to bring the platform-specific metrics closer to the Notes-specific metrics, so the information can be monitored and analyzed from one process. For example, you'll be able to display Domino statistics side-by-side with OS metrics like 600 users using a defined amount of memory, with a specified CPU and disk utilization.
What is the database size limit? How will this affect performance?
James Grigsby
Databases are now certified up to 64GB. Other IBM platforms may certify even higher per database size limits. While the individual certified file size is 64GB, development tests run with 50,000+ databases on a single server. The database size has a minimal impact on performance.
What kinds of view/update and view/rebuild enhancements can we expect?
James Grigsby
The view updates and rebuilds are now three times faster than for R4.6.
What kind of Iris support resources are there? Where can we go on the Web for information on Domino performance issues?
Carol Zimmet
Of course, you can use Notes.net, where there are several performance articles. We will also be answering questions again in the Iris Cafe during the month of August. If there are performance issues you want us to focus on, this is a great place to post requests.
In addition, there's also the NotesBench Consortium, which offers the Server.Planner datasets as well as published vendor reports, and a general exchange of information and questions between users. And there's a new section on the Lotus Web site called the Domino Performance Zone.
What about products from Lotus Business Partners - are there specific companies we should look for?
Carol Zimmet
Yes, there are a number of companies that have introduced performance planning products, and we encourage people to look for them by going to the Business Partners section of the Lotus Web site.
What performance areas are you concentrating on for post-R5 - what's your top hit list?
James Grigsby
I think that we'll see more 64-bit performance work, especially as the 64-bit Intel-based hardware, 64-bit compilers, and 64-bit add-ins become available. As always, new usage models will evolve like Knowledge Management, synchronous messaging, and cross-product workflow. That's really the fun part. We get to define and shape these new models that are more customer-friendly. Just look at the success of the NotesBench Consortium (with 10,000+ registered users in one year) and Ideas International (tracking our benchmarks with the industry standards). It says we're doing something right to help ease the administrator's decision load.
The official Lotus Notes Benchmark tool - NotesBench. Lotus Server.Load (a free GUI Notes load generation tool) is available on the Domino R5.0.1 CD and on the Lotus Performance Zone Web site]
|