As a tools developer for CA Gen, the fact that we also develop our tools with CA Gen has meant that we tend to 'build not buy'. In other words, when we find the need for some additional tools support as part of our own Gen development projects, we enhance our own tools. This approach then leads to extra functionality in the products that is almost always useful to our customers as well.
However there are certain 3rd party tools and utilities that we have purchased runtime licences for instead of building ourselves. Examples include the diagramming OCX that we have used for creating the Object Structure, Life-Cycle and Model Architecture diagrams in GuardIEn, the ftp/sftp utilities and the file compare tool.
For the file compare tool, we have used the freely distributable WinDiff tool with the option for customers to replace this with their own favourite product. However Windiff is a fairly basic tool, and some time ago we replaced this for internal use with Beyond Compare 3 (BC3).
We like BC3 so much, that for the 8.0 release of our products, we have purchased additional licences to be able to distribute BC3 to our customers as well.
Tuesday, 4 May 2010
Wednesday, 28 April 2010
Gen r8 and z/OS
The beta test for Gen r8 has now ended and we are finishing off the changes made to our products to support r8. We will be launching this release 8.0 in early May to coincide with the general availability of Gen 8.0.
The most significant changes that affected us were on the z/OS platform. The introduction of z/OS Libraries (OPSLIBs for z/OS), dynamic RI triggers and changes to the way that applications are linked affected many aspects of GuardIEn, especially in the areas of impact analysis and code construction.
In previous releases of Gen, the link-edit control cards were created from the Gen skeletons using file tailoring and then a single INCLUDE was added for the dialog manager, with the remaining modules included using autocall.
With Gen r8, the format of the link-edit control cards has changed. Instead of using autocall to resolve called action blocks, each non-compatibility Gen module referenced in the load module has a specific INCLUDE APPLOAD or IMPORT statement.
This means that if you create the link-edit control cards outside of Gen, you will have to replicate this approach. A new business system library is available which Gen populates with the link-edit control cards (called binder control cards using the new IBM terminology), so these are available if required.
Another change is that dynamic action blocks that are packaged into a z/OS Lib are now called using a literal instead of a variable, for example, in Gen r7 and previously, a call to action block AAAA would be implemented as:
09 AAAA-ID PIC X(8) VALUE 'AAAA'.
...
CALL AAAA-ID
In Gen r8, if AAAA is included in a z/LIB, this is now
CALL 'AAAA'
If you installing code using multiple models, then the use of external action block and external system load libraries must be carefully considered to ensure that dynamic action blocks packaged into a z/LIB are not found via autocall since the binder will then statically link the object modules instead of resolving them using the IMPORT statement.
The most significant changes that affected us were on the z/OS platform. The introduction of z/OS Libraries (OPSLIBs for z/OS), dynamic RI triggers and changes to the way that applications are linked affected many aspects of GuardIEn, especially in the areas of impact analysis and code construction.
In previous releases of Gen, the link-edit control cards were created from the Gen skeletons using file tailoring and then a single INCLUDE was added for the dialog manager, with the remaining modules included using autocall.
With Gen r8, the format of the link-edit control cards has changed. Instead of using autocall to resolve called action blocks, each non-compatibility Gen module referenced in the load module has a specific INCLUDE APPLOAD or IMPORT statement.
This means that if you create the link-edit control cards outside of Gen, you will have to replicate this approach. A new business system library is available which Gen populates with the link-edit control cards (called binder control cards using the new IBM terminology), so these are available if required.
Another change is that dynamic action blocks that are packaged into a z/OS Lib are now called using a literal instead of a variable, for example, in Gen r7 and previously, a call to action block AAAA would be implemented as:
09 AAAA-ID PIC X(8) VALUE 'AAAA'.
...
CALL AAAA-ID
In Gen r8, if AAAA is included in a z/LIB, this is now
CALL 'AAAA'
If you installing code using multiple models, then the use of external action block and external system load libraries must be carefully considered to ensure that dynamic action blocks packaged into a z/LIB are not found via autocall since the binder will then statically link the object modules instead of resolving them using the IMPORT statement.
Wednesday, 31 March 2010
Multi Row Fetch
One of the new features in Gen r8 that we have started to test is the ability to specify a multi-row fetch for READ EACH statements. This property changes the generated code to perform a block fetch of multiple rows into an array rather than fetching one row at a time.
A simple test of reading many rows from a large table using multi row fetch provided a 60% reduction in CPU compared with a standard READ EACH table. (The test was a DB2/COBOL batch job without much code apart from the READ EACH, so a real world example will show less of a gain)
The test compared various array sizes of 100, 1000 and 10000. There was very little difference in the cpu consumed between these, showing that there seems to be little benefit in setting a very large array size.
When we upgrade our own code to using Gen r8 for the next release, we plan to update all READ EACH statements to use this new option. To help do this, we have developed a new VerifIEr check that ensures that all READ EACH statements have this option specified and a corresponding genIE automated fix to set the property value. This intelligently inspects the code within the READ EACH to determine what array size to use, and it can also have an upper limit defined.
A simple test of reading many rows from a large table using multi row fetch provided a 60% reduction in CPU compared with a standard READ EACH table. (The test was a DB2/COBOL batch job without much code apart from the READ EACH, so a real world example will show less of a gain)
The test compared various array sizes of 100, 1000 and 10000. There was very little difference in the cpu consumed between these, showing that there seems to be little benefit in setting a very large array size.
When we upgrade our own code to using Gen r8 for the next release, we plan to update all READ EACH statements to use this new option. To help do this, we have developed a new VerifIEr check that ensures that all READ EACH statements have this option specified and a corresponding genIE automated fix to set the property value. This intelligently inspects the code within the READ EACH to determine what array size to use, and it can also have an upper limit defined.
Thursday, 11 February 2010
Build Tool Profile - Oracle
We have noticed that the Gen Build Tool uses different Oracle pre-compiler options for UNIX vs Windows. Here are the default options:
Windows:
SQLCHECK=SYNTAX MODE=ANSI IRECLEN=255 ORECLEN=255 LTYPE=NONE
UNIX:
IRECLEN=511 ORECLEN=511 CODE=ANSI_C LTYPE=NONE MODE=ANSI DBMS=V7
SQLCHECK=SYNTAX PARSE=NONE RELEASE_CURSOR=NO HOLD_CURSOR=YES
We are not sure why RELEASE_CURSOR=NO and HOLD_CURSOR=YES are not set for Windows. We have experimented with setting these for Windows and have noticed a significant performance improvement.
Windows:
SQLCHECK=SYNTAX MODE=ANSI IRECLEN=255 ORECLEN=255 LTYPE=NONE
UNIX:
IRECLEN=511 ORECLEN=511 CODE=ANSI_C LTYPE=NONE MODE=ANSI DBMS=V7
SQLCHECK=SYNTAX PARSE=NONE RELEASE_CURSOR=NO HOLD_CURSOR=YES
We are not sure why RELEASE_CURSOR=NO and HOLD_CURSOR=YES are not set for Windows. We have experimented with setting these for Windows and have noticed a significant performance improvement.
Monday, 25 January 2010
Application Cache
If you frequently need to read the same data from the database in the same transaction (for example look-up table data), the DBMS cache/buffer should reduce the I/O to disk by having the data in memory. However there is still a considerable overhead involved with using SQL to obtain the data multiple times.
A technique that we have used to improve performance is an application cache. This uses a common action block to read the data. The action block checks to see if the data is cached, and if it is, avoids the need for a READ from the database. If it is not available, it reads the data and then stores it in the cache.The cache is implemented using uninitialised local views and you can store as many rows as you are prepared to allocate memory to.
The design needs to ensure that the cache is initialised at the start of the transaction (for C generated code when using the TE) and also to cater for the possibility of updates to the database, i.e. the cache needs refreshing or deleting if the data can be changed during the transaction.
In specific examples where we have used the cache for heavily accessed tables, we have found the cache improves the performance of the READ action block by 1000%.
A technique that we have used to improve performance is an application cache. This uses a common action block to read the data. The action block checks to see if the data is cached, and if it is, avoids the need for a READ from the database. If it is not available, it reads the data and then stores it in the cache.The cache is implemented using uninitialised local views and you can store as many rows as you are prepared to allocate memory to.
The design needs to ensure that the cache is initialised at the start of the transaction (for C generated code when using the TE) and also to cater for the possibility of updates to the database, i.e. the cache needs refreshing or deleting if the data can be changed during the transaction.
In specific examples where we have used the cache for heavily accessed tables, we have found the cache improves the performance of the READ action block by 1000%.
Wednesday, 13 January 2010
Mapping Group Views
Gen allows you to view map group views with differing cardinalities on USE statements and dialog flows if the receiving view has a higher cardinality than the sending view.
However the view match remains intact even if you then subsequently change the cardinality of the sending view to a value that is greater than the receiving view, so you could end up with a sending view that is larger than the receiving view, which could then cause unexpected results, like loss of data without a runtime error. In this case, you could not establish the view match again, but the existing view match is still 'valid' in the model.
If the group view sizes were initially the same, the developer might not think that they need to add in any extra validation logic, but a subsequent change to one of the group views might then cause problems.
A new check in VerifIEr allows a quick check for differing group view cardinalities with a warning if they differ but are valid and an error if they differ and are invalid.
However the view match remains intact even if you then subsequently change the cardinality of the sending view to a value that is greater than the receiving view, so you could end up with a sending view that is larger than the receiving view, which could then cause unexpected results, like loss of data without a runtime error. In this case, you could not establish the view match again, but the existing view match is still 'valid' in the model.
If the group view sizes were initially the same, the developer might not think that they need to add in any extra validation logic, but a subsequent change to one of the group views might then cause problems.
A new check in VerifIEr allows a quick check for differing group view cardinalities with a warning if they differ but are valid and an error if they differ and are invalid.
Monday, 21 December 2009
Discovering hidden errors
Since we originally developed our VerifIEr QA and code checking product over 2 years ago, the scope of checks has enlarged in some very interesting directions.
The initial checks were primarily aimed at standards enforcement, for example, object names, CBD architecture enforcement, use (or abuse) of various properties (e.g. READ properties), etc.
More recently however we have developed a number of checks for customers that are aimed at improving code quality by detecting errors that might be otherwise difficult to find.
We have run these checks on our own products (which are developed with Gen) and have been surprised at the number of potential errors that have been encountered. Usually the more serious errors are detected during testing, but sometimes not!
Examples of the checks that we have found especially useful include:
The initial checks were primarily aimed at standards enforcement, for example, object names, CBD architecture enforcement, use (or abuse) of various properties (e.g. READ properties), etc.
More recently however we have developed a number of checks for customers that are aimed at improving code quality by detecting errors that might be otherwise difficult to find.
We have run these checks on our own products (which are developed with Gen) and have been surprised at the number of potential errors that have been encountered. Usually the more serious errors are detected during testing, but sometimes not!
Examples of the checks that we have found especially useful include:
- Local views that are referenced but never set to a value, indicating that either the local views need to be populated or the code is no longer required;
- Hidden fields (for example fields on a GUI that are placed above the top border) that are not read-only (the user could therefore tab to the field and change its value);
- Export views not fully mapped to an import view on a screen or window;
- Checking the tab sequence for GUI windows and dialog boxes;
These sorts of errors are notoriously difficult to spot via code inspection and can also be missed during testing.
Wednesday, 9 December 2009
Gen r8 Beta Testing
October and November were busy months with 3 Edge conferences to attend and the start of the Gen r8 beta test period, so apologies for no blog posting for the past few weeks.
We have started our beta testing of the next Gen release (r8). There are a large number of new and changed features that will keep us busy with the development of the 8.0 release of our products that will support Gen r8.
A number of customers have asked about the availability of IET products to support their beta testing of Gen r8. Unfortunately we have only been able to start development of Release 8.0 of our products with the start of the beta programme and hence cannot offer a beta version of our products until this development work is complete.
We have started our beta testing of the next Gen release (r8). There are a large number of new and changed features that will keep us busy with the development of the 8.0 release of our products that will support Gen r8.
A number of customers have asked about the availability of IET products to support their beta testing of Gen r8. Unfortunately we have only been able to start development of Release 8.0 of our products with the start of the beta programme and hence cannot offer a beta version of our products until this development work is complete.
Thursday, 22 October 2009
Unused and Duplicate Prompts
We have been developing our products for close to 20 years now and one of the consequences has been that we have found quite a few unused and duplicate prompts in the models. We also have a multiple model architecture and a policy of migrating the entire data model to each of the development models. This results in all of the prompts also being duplicated (and unused) in all of the models.
Apart from having a large number of redundant prompts in the models, it can also make the selection of prompts in the window/screen designer tedious because the large lists of unused and duplicate prompts makes locating the desired prompt harder.
There is a Gen function in the toolset to delete unused prompts, but this requires the model to be downloaded, and ours are too big. It will also not get rid of duplicates.
We have therefore written a new genIE function to both delete unused and consolidate duplicate prompts.
The result is faster downloads because you are not downloading extra prompts and also easier selection of existing prompts in the window/screen designer.
Apart from having a large number of redundant prompts in the models, it can also make the selection of prompts in the window/screen designer tedious because the large lists of unused and duplicate prompts makes locating the desired prompt harder.
There is a Gen function in the toolset to delete unused prompts, but this requires the model to be downloaded, and ours are too big. It will also not get rid of duplicates.
We have therefore written a new genIE function to both delete unused and consolidate duplicate prompts.
The result is faster downloads because you are not downloading extra prompts and also easier selection of existing prompts in the window/screen designer.
Thursday, 1 October 2009
RI Trigger Impact Analysis
The previous post discussed the need to regenerate programs that call changed RI triggers. The difficulty is in performing this impact analysis.
The Gen model does contain associations between action blocks and RI triggers (using IMPUSE associations if you are familiar with the Gen schema), but the important point to note is that these associations are maintained by the code generators. This means that if you change the data model, the IMPUSE association data in the model is not accurate until you have regenerated the affected code, which makes it useless in helping you understand what you need to regenerate!
We faced this issue when developing the impact analysis process that GuardIEn performs when it detects data model changes. To work out what RI triggers are directly affected by a data model change is straight forward, but the consequential impact on other triggers and action block / procedure steps involves a complex navigation though the data model, following cascade delete chains for example. It is also affected by the choice of generated or DBMS enforced RI rules.
The Gen model does contain associations between action blocks and RI triggers (using IMPUSE associations if you are familiar with the Gen schema), but the important point to note is that these associations are maintained by the code generators. This means that if you change the data model, the IMPUSE association data in the model is not accurate until you have regenerated the affected code, which makes it useless in helping you understand what you need to regenerate!
We faced this issue when developing the impact analysis process that GuardIEn performs when it detects data model changes. To work out what RI triggers are directly affected by a data model change is straight forward, but the consequential impact on other triggers and action block / procedure steps involves a complex navigation though the data model, following cascade delete chains for example. It is also affected by the choice of generated or DBMS enforced RI rules.
Subscribe to:
Posts (Atom)