Over the years a number of CSE sites have tried to place the CSE database and software on separate physical servers. Technically there is no issue with making this function correctly, but what we have seen is that the additional network traffic required slows down the overall performance and response times of the software considerably, the point where it outweighs the benefits of spreading the CPU load. This is especially noticeable for servers that have huge amounts of memory, where large portions of the database are available in cache, and therefore the network response becomes a larger percentage of the overall transaction since disk i/o is less of a bottleneck.
Where customers have utilised this approach, it has been necessary to implement complex and expensive dedicated network pipes between the servers, and frankly is not worth it just to spread the load, where actually a single powerful server is perfectly capable of serving the needs of the entire Gen development team.
We therefore strongly recommend that the database and software reside on the same machine.
Showing posts with label Encyclopaedia Performance. Show all posts
Showing posts with label Encyclopaedia Performance. Show all posts
Monday, 20 February 2012
Wednesday, 1 September 2010
Parallel Generation Results
We have now implemented the new CSE server. This has two 4-core processors and we recently conducted a simple test to benchmark the improvements gained when running CSE generation tasks in parallel (See previous post for introduction).
The results were that we obtained was a 60% reduction in elapsed time when running 4 threads in parallel and 70% reduction for 8 threads.
This was obtained with no other processes running, so for normal use, we plan to restrict a single generate task to a maximum of 4 parallel generation threads because of other tasks and on-line server processing requirements.
The results were that we obtained was a 60% reduction in elapsed time when running 4 threads in parallel and 70% reduction for 8 threads.
This was obtained with no other processes running, so for normal use, we plan to restrict a single generate task to a maximum of 4 parallel generation threads because of other tasks and on-line server processing requirements.
Friday, 28 May 2010
Encyclopaedia Performance
A few customers have asked us for information on improving performance on both the Gen host ency and client/server ency. Rather than try and write out a definitive list of things to look for, I thought it would be useful (and less onerous) to provide occasional posts on this subject.
If you have any ideas or experiences that you would like to share, then please add comments to the postings, or if you prefer, e-mail them and they can be included in a new post.
The areas that will be considered in this section include encyclopaedia server performance, reducing contention, working practices and other factors that affect the overall performance of the encyclopaedia.
To start things off:
HE & CSE:
1) Model size rather than overall ency size has a big impact on performance. Smaller models perform much better than larger ones for the same function, i.e. downloading the same subset from a small model will be much faster than a large model. The speed is roughly proportional to the model size.
2) Encourage 'right sizing' subsets and working practices that are efficient, like only generating changed objects rather than all action blocks in a load module.
CSE:
1) Use the support client and check the Object Cache setting. The default is far too low. We have ours set to 500,000
2) Database i/o has a major impact on performance. Does your DBMS have enough memory for caching? Oracle likes a lot!
HE:
Have you implemented DB2 type 2 indices for the encyclopaedia? Originally (many years ago now) these were not available for DB2, so if you have an old encyclopaedia, you may not have converted the indices to type 2. This can have a big impact on contention. Type 1 indices are not supported from DB2 v8 onwards, so if you are on v8, then this will have been taken care of.
If you have any ideas or experiences that you would like to share, then please add comments to the postings, or if you prefer, e-mail them and they can be included in a new post.
The areas that will be considered in this section include encyclopaedia server performance, reducing contention, working practices and other factors that affect the overall performance of the encyclopaedia.
To start things off:
HE & CSE:
1) Model size rather than overall ency size has a big impact on performance. Smaller models perform much better than larger ones for the same function, i.e. downloading the same subset from a small model will be much faster than a large model. The speed is roughly proportional to the model size.
2) Encourage 'right sizing' subsets and working practices that are efficient, like only generating changed objects rather than all action blocks in a load module.
CSE:
1) Use the support client and check the Object Cache setting. The default is far too low. We have ours set to 500,000
2) Database i/o has a major impact on performance. Does your DBMS have enough memory for caching? Oracle likes a lot!
HE:
Have you implemented DB2 type 2 indices for the encyclopaedia? Originally (many years ago now) these were not available for DB2, so if you have an old encyclopaedia, you may not have converted the indices to type 2. This can have a big impact on contention. Type 1 indices are not supported from DB2 v8 onwards, so if you are on v8, then this will have been taken care of.
Subscribe to:
Posts (Atom)