Over the years a number of CSE sites have tried to place the CSE database and software on separate physical servers. Technically there is no issue with making this function correctly, but what we have seen is that the additional network traffic required slows down the overall performance and response times of the software considerably, the point where it outweighs the benefits of spreading the CPU load. This is especially noticeable for servers that have huge amounts of memory, where large portions of the database are available in cache, and therefore the network response becomes a larger percentage of the overall transaction since disk i/o is less of a bottleneck.
Where customers have utilised this approach, it has been necessary to implement complex and expensive dedicated network pipes between the servers, and frankly is not worth it just to spread the load, where actually a single powerful server is perfectly capable of serving the needs of the entire Gen development team.
We therefore strongly recommend that the database and software reside on the same machine.
Monday, 20 February 2012
Friday, 17 February 2012
z/OS Operations Libraries and Dynamic Linking
Gen r8 introduced a new feature to package action blocks into z/OS Operations Libraries (z/LIB). These are DLLs that contain one or more action blocks and are similar in concept to the operations libraries that have been available in Gen for the Windows and UNIX environments for some time.
One difference between Windows/UNIX and z/OS relates to the use of the MVS Dynamic Linking property of an action block (or business system default). To be eligible for packaging into a z/LIB, the action block must be marked as dynamic (not static or compatibility).
You should be careful when converting from purely dynamic action blocks to z/LIBs. If you only package the top level action blocks into the z/LIB, the lower level action blocks will be called dynamically from a separate load module if they remain dynamic and are not packaged into the z/LIB.
Consider the following example: AB1 calls AB2 and both are marked as dynamic, either as a business system default or explicitly.
If you create a z/LIB and only add AB1 into the z/LIB, since AB2 is private to AB1, then the z/LIB will only contain AB1 and AB2 remains an 'old style' dynamic action block with its own load module and the call from AB1 to AB2 is dynamic.
If AB2 is private, it should either be changed to static so that it is only statically linked into the z/LIB, or added to the scope of z/LIB so that it is also linked into the z/LIB.
One difference between Windows/UNIX and z/OS relates to the use of the MVS Dynamic Linking property of an action block (or business system default). To be eligible for packaging into a z/LIB, the action block must be marked as dynamic (not static or compatibility).
You should be careful when converting from purely dynamic action blocks to z/LIBs. If you only package the top level action blocks into the z/LIB, the lower level action blocks will be called dynamically from a separate load module if they remain dynamic and are not packaged into the z/LIB.
Consider the following example: AB1 calls AB2 and both are marked as dynamic, either as a business system default or explicitly.
If you create a z/LIB and only add AB1 into the z/LIB, since AB2 is private to AB1, then the z/LIB will only contain AB1 and AB2 remains an 'old style' dynamic action block with its own load module and the call from AB1 to AB2 is dynamic.
If AB2 is private, it should either be changed to static so that it is only statically linked into the z/LIB, or added to the scope of z/LIB so that it is also linked into the z/LIB.
Tuesday, 24 January 2012
Static Code Analysis
Of the 150 checks currently available in VerifIEr, a good many are aimed at detecting errors in code. If you can find an error in the code prior to code generation, it is much easier and quicker to fix, and fixes an error that might not get detected during testing.
Until recently my favourite check was one that detects views that are used but not populated, since this usually indicates an error.
My new favourite is our latest check, which looks for ambiguous OR clauses in an expression. For example, the following code is ambiguous: IF x=1 OR x=2 AND y=3 and should be written as IF (x=1 OR x=2) AND y=3.
We ran this check on our current code base and a few clauses were highlighted. As an example of how difficult it can be to spot these errors, can anyone see the error in this READ statement? We didn't, but the check did!
Until recently my favourite check was one that detects views that are used but not populated, since this usually indicates an error.
My new favourite is our latest check, which looks for ambiguous OR clauses in an expression. For example, the following code is ambiguous: IF x=1 OR x=2 AND y=3 and should be written as IF (x=1 OR x=2) AND y=3.
We ran this check on our current code base and a few clauses were highlighted. As an example of how difficult it can be to spot these errors, can anyone see the error in this READ statement? We didn't, but the check did!
Thursday, 5 January 2012
Unadopting action blocks
A customer recently contacted us with an issue related to having two action blocks (in different models) that had the same ancestry but were otherwise different, i.e. different names, logic, source code member name, etc. Because they had the same ancestry, GuardIEn was managing them as the same deliverable, but they needed to be managed separately.
The solution was to give the second copy of the action block different ancestry, but how, since there isn't an unadopt function on the CSE.
The workaround was as follows:
The solution was to give the second copy of the action block different ancestry, but how, since there isn't an unadopt function on the CSE.
The workaround was as follows:
- scope a subset containing the ABs that you want to unadopt
- download the subset
- generate a new (temp) model on the CSE from the subset so that the ABs in the new model have new ancestry
- selectively adopt just the business systems in the new model to the old model
- selectively adopt the ABs in the old model to the ABs in the new model so that the ancestry in the old model changes
- delete the new (temp) model
Thursday, 8 December 2011
View Starving or Perfect View Matching?
A recent question from a customer was related to view starving and perfect view matching.
The technique of view starving involves removing all redundant attributes from views to avoid unnecessary storage allocation and view initialisation logic.
The technique of perfect view matching involves ensuring that the view structures in a USE statement are identical so that the program call can be achieved via passing of the actual views.
The question related to the problem that if you starve the views, you might not get perfect view matching, and hence should you then allocate a new local view that has perfect view matching and add in extra statements to manually maintain this temporary view to achieve perfect view matching for the USE statement.
If you do not have perfect view matching, then Gen needs to generate an intermediate data structure in the calling program to accomplish the parameter passing on the call of the used action block.
Data needs to then be passed from the actual views in the calling AB to the intermediate data structure and therefore there is an overhead for a) allocating the storage for the intermediate structures and b) the instructions to move the data.
Therefore in this situation, there is little difference between doing this yourself via a local view or getting Gen to automatically generate the extra structures – it amounts to the same thing, at least for one USE statement. Note that the order of the views is also important. Your local view would have to have the same attributes in the same order, and to work this out requires a careful comparison of the two views since the view matching dialog will not indicate if they are ordered differently.
It only becomes more efficient to define your own views if you have many USE statements that would make use of your fully populated view, since Gen would generate the data moves for each USE whereas you may only need to move the data once for multiple calls. However this becomes more complicated to understand and maintain. It only needs one attribute to be added, deleted or even moved in sequence for the technique to not accomplish perfect view matching, and then you get a double overhead – your views and the extra generated code. The additional code also complicates the action diagram, often for little benefit.
In an on-line transaction with a single USE statement, the overhead is not worth worrying about. You should concentrate on achieving perfect view matching where it will affect performance. This would typically be for large group views and repeated calls to the same action block, for example, in batch jobs where the same action block is called within a loop that is executed many times.
With our automated code checking tool VerifIEr, we have a perfect view matching check. This will work out whether you have perfect view matching or not, and can be configured to only check group views and USEs within loops, so that you only have to focus on the important ones. VerifIEr can also check that attributes are used, and hence help with view starving.
Where you want to have perfect view matching for performance reasons, I would add in the extra attributes to the ‘proper’ views rather than add in extra views, since there would be no benefit from adding in the extra views. The only time this might be needed is if there are many called ABs with different import views and all sourced from the same view in the caller. However in this situation I would then let Gen handle this situation rather than introduce my own additional views.
The technique of view starving involves removing all redundant attributes from views to avoid unnecessary storage allocation and view initialisation logic.
The technique of perfect view matching involves ensuring that the view structures in a USE statement are identical so that the program call can be achieved via passing of the actual views.
The question related to the problem that if you starve the views, you might not get perfect view matching, and hence should you then allocate a new local view that has perfect view matching and add in extra statements to manually maintain this temporary view to achieve perfect view matching for the USE statement.
If you do not have perfect view matching, then Gen needs to generate an intermediate data structure in the calling program to accomplish the parameter passing on the call of the used action block.
Data needs to then be passed from the actual views in the calling AB to the intermediate data structure and therefore there is an overhead for a) allocating the storage for the intermediate structures and b) the instructions to move the data.
Therefore in this situation, there is little difference between doing this yourself via a local view or getting Gen to automatically generate the extra structures – it amounts to the same thing, at least for one USE statement. Note that the order of the views is also important. Your local view would have to have the same attributes in the same order, and to work this out requires a careful comparison of the two views since the view matching dialog will not indicate if they are ordered differently.
It only becomes more efficient to define your own views if you have many USE statements that would make use of your fully populated view, since Gen would generate the data moves for each USE whereas you may only need to move the data once for multiple calls. However this becomes more complicated to understand and maintain. It only needs one attribute to be added, deleted or even moved in sequence for the technique to not accomplish perfect view matching, and then you get a double overhead – your views and the extra generated code. The additional code also complicates the action diagram, often for little benefit.
In an on-line transaction with a single USE statement, the overhead is not worth worrying about. You should concentrate on achieving perfect view matching where it will affect performance. This would typically be for large group views and repeated calls to the same action block, for example, in batch jobs where the same action block is called within a loop that is executed many times.
With our automated code checking tool VerifIEr, we have a perfect view matching check. This will work out whether you have perfect view matching or not, and can be configured to only check group views and USEs within loops, so that you only have to focus on the important ones. VerifIEr can also check that attributes are used, and hence help with view starving.
Where you want to have perfect view matching for performance reasons, I would add in the extra attributes to the ‘proper’ views rather than add in extra views, since there would be no benefit from adding in the extra views. The only time this might be needed is if there are many called ABs with different import views and all sourced from the same view in the caller. However in this situation I would then let Gen handle this situation rather than introduce my own additional views.
Friday, 18 November 2011
Trace Lite
When you can't work out what is going wrong with your code, you can often solve the issue by stepping through the statements using the Diagram Trace Utility (or xTrace on the z/OS platform). However this requires regeneration of the code with the trace option enabled and this takes a bit of time.
In many cases, just seeing which statements were (or weren't) executed is enough to locate the cause of the problem. You can now do this if you are using pathvIEw, since it has a feature to display the last executed statements.
In many cases, just seeing which statements were (or weren't) executed is enough to locate the cause of the problem. You can now do this if you are using pathvIEw, since it has a feature to display the last executed statements.
Tuesday, 25 October 2011
Moving beyond the 32k CFB limit
Gen r8 IE2 increases the Common Format Buffer (CFB) limit from 32k to 16M.
The increase in the CFB limit is available for C generated applications on Windows and UNIX, but not in this release for z/OS, and hence we will not be able to take advantage of this for our products until z/OS support is available for CICS and IMS COBOL servers. It must be the top ranked enhancement request for CA Gen for the past 20+ years, so it is great to see this finally make it into the product.
Previous posts have discussed strategies for coping with the 32k view size limit, and our experiences with a Web View interface has been that it is a good idea for web applications to fetch and display small pages of data rather than trying to bring back a huge results set into a group view. The 32k limit therefore can be a good thing because it provides a limit on the amount of data returned in a single server call.
With the new limit, a developer could massively increase the data returned by a server to ~16M, which might be a good thing if the application did this anyway with repeated server calls, but a bad thing if the user was previously responsible for paging through the data and using filter/selection fields to limit the number of rows displayed, but they can now display thousands of rows without needing to worry about the filters.
Thinking about the impact on our own products (which are developed with Gen), there are several servers that would be much simpler with a larger export view size, so once z/OS support is available, we will take advantage of the larger view sizes to simplify the code and improve performance.
The increase in the CFB limit is available for C generated applications on Windows and UNIX, but not in this release for z/OS, and hence we will not be able to take advantage of this for our products until z/OS support is available for CICS and IMS COBOL servers. It must be the top ranked enhancement request for CA Gen for the past 20+ years, so it is great to see this finally make it into the product.
Previous posts have discussed strategies for coping with the 32k view size limit, and our experiences with a Web View interface has been that it is a good idea for web applications to fetch and display small pages of data rather than trying to bring back a huge results set into a group view. The 32k limit therefore can be a good thing because it provides a limit on the amount of data returned in a single server call.
With the new limit, a developer could massively increase the data returned by a server to ~16M, which might be a good thing if the application did this anyway with repeated server calls, but a bad thing if the user was previously responsible for paging through the data and using filter/selection fields to limit the number of rows displayed, but they can now display thousands of rows without needing to worry about the filters.
Thinking about the impact on our own products (which are developed with Gen), there are several servers that would be much simpler with a larger export view size, so once z/OS support is available, we will take advantage of the larger view sizes to simplify the code and improve performance.
Gen 8 Interim Enhancement 2
We have just started beta testing Gen r8 Interim Enhancement 2 (IE2).
(Interim Enhancement is the new term within CA for Feature Pack).
The new features that we are particularly interested in are:
(Interim Enhancement is the new term within CA for Feature Pack).
The new features that we are particularly interested in are:
- Increase in the common format buffer (CFB) limit from 32k to 16M
- 64-bit Windows applications
- Customised Java proxies
Tuesday, 30 August 2011
Visualising Data
With the development of pathvIEw, we wanted to provide a graphical display of how complete the testing was for each module. To do this, we developed a new function in IETeGUI to provide a percentage complete indicator to display in a listbox cell, like this: 
This got us thinking about other ways of visualising data to make it easy to glance at a table of data and see the most important bits. We therefore created two additional bar chart styles which are illustrated below. The first style is just a single bar and is used to illustrate the number of statements - the longer the bar, the more statements there are.
The second style is similar to the first, except that the colour is based on a medium and high threshold. We have used this for the complexity column, with green showing low complexity values, amber for medium complexity and red for high complexity.
The nice part about using IETeGUI is that the above is handled without the need for any additional OCX controls and is displayed in a standard Gen listbox.
Friday, 26 August 2011
Cyclomatic Complexity and CA Gen
As part of the development of pathvIEw, we have introduced a new complexity metric which is calculated for each action block and procedure step.
We conducted some research into the usefulness of the metric and this shows some interesting results into the use of complexity counts for predicting error rates in code and hence for its use in prioritising testing efforts.
See the research document here: Cyclomatic Complexity and CA Gen.
We conducted some research into the usefulness of the metric and this shows some interesting results into the use of complexity counts for predicting error rates in code and hence for its use in prioritising testing efforts.
See the research document here: Cyclomatic Complexity and CA Gen.
Subscribe to:
Posts (Atom)