Web View Experiences - View Sizes

Our GUI client design for list boxes is to retrieve all of the rows that satisfy the filter criteria and then allow the user to easily scroll up and down through the list box. Because of the 32k view size limit on the server procedure step, the client list box size is much larger than the server view size. For example, the client list box might allow up to 32000 rows (the limit for a GUI list box is 32767 rows) but the server group view size might be 100 rows, so the client would call the server multiple times to retrieve all of the data.

Whilst this approach works well for a GUI client, it presents performance problems for the web interface. Therefore our design for the web interface is to only call the server once to retrieve the first set of rows (for example 100) and then provide Prev and Next buttons on the web page to support paging.

The servers will already support the Next page operation because this would use a 'start from' field that would be set in the GUI client to page forward through the data. However our servers typically did not need to support a 'previous page' operation and so this has had to be added to the servers to support the web interface.

Web View Experiences - Introduction

In the latest release of GuardIEn (8.1) that we launched this month, we introduced a web based user interface for viewing and managing Change Requests and Release Packs. Called WebCR, the primary purpose for developing this new interface was to provide an easy way of giving a wider group of users access to the change management functions of GuardIEn without needing to install the full client software on their workstation.

All of our products are developed with CA Gen, and we also wanted to develop the web interface using Gen so that we could maintain all of our code in a single tool, and share common logic between the GUI client and web interfaces.

In the past we had experimented with Gen's Web Generation capabilities, but decided that these did not meet our requirements. However with the introduction of Web View in Gen 8.0, we decided to re-evaluate Gen's capabilities and started an R&D project with Web View. The results were promising and the decision was taken to proceed with the full scale development of a new web based user interface for the change management aspects of GuardIEn.

The design criteria was to provide similar functionality as the GUI client and use the same servers as the GUI, so that a customer wanting to use WebCR simply needs to install the WebCR application in an application server and the user interface is up and running.

Whilst we wanted to provide similar functionality to the GUI interface, our research project indicated that there were some important differences between GUI clients and Web clients with respect to dialog design, window layout and performance. This meant that we needed to have separate client procedure steps for the web interface, although they could share the same servers as the GUI client and also common code.

In subsequent posts, I will describe some of our experiences of using Web View in more detail.

RI Triggers – Gen or DBMS?

Sorry for the delay since the last posting, but we have been very busy at IET with launching a new release along with a couple of new products.

We recently had a discussion with a customer regarding the difference between Gen and DBMS RI enforcement.

The advantage of using DBMS RI is that the RI integrity is maintained by the database and not Gen generated code and hence any program or interactive SQL that deletes records will ensure RI integrity is maintained, whereas with Gen RI, you must either always perform deletes using Gen programs or ensure that your non-Gen programs or SQL correctly maintain RI integrity by cascade deleting child rows, setting foreign keys to NULL, etc.

However one important consideration is that many DBMS products do not support the full range of delete rules that can be defined in Gen. One example is a pendant delete, where the parent row is deleted when the last child is deleted. In this situation, Gen will enforce the rules that cannot be enforced by the DBMS, so that the generated RI triggers contain a mixture of Gen and DBMS enforced rules.

The danger with this situation is that you might think that all RI is enforced by the DBMS and hence not worry about deletes performed outside of Gen, however the DBMS would only be performing some of the deletes and hence the results would differ between using Gen to perform a delete compared with non-Gen programs.

Another consideration with DBMS RI is that you must ensure that the DBMS rules are kept up to date, on all databases, for example, development, test and production.

For these reasons, we use Gen enforced RI for our products.

GuardIEn and Versions

I was recently asked to clarify how GuardIEn handles code versions, and thought that posting the reply might be helpful to others.

GuardIEn stores meta-data about versions (i.e. information about the versions) rather than the versions themselves, and the actual versions of objects have to be stored as objects in Gen models. Hence to be able to ‘use’ any prior version in a Gen process like a migrate, it must exist in a model.

If you consider an example of a 3-level development hierarchy with DEV, TEST and PROD environments, then as long as you have a Gen model associated to each environment, you could have up to 3 separate versions of an object, one per model.

• If you start with the same copy of the object in each and assign it Version 1, then in GuardIEn there will be one version (V1) and that version is in the DEV, TEST & PROD models.
  
• You change the object in DEV and upload it using the Upload Assistant, GuardIEn will create a new version (V2) and assign it to the first state in the life-cycle for the DEV environment. You then have two versions in GuardIEn and two versions in the Gen encyclopaedia – V1 in TEST & PROD and V2 in DEV.
  
• You migrate the object from DEV to TEST and the status of V2 will be updated to a TEST state and you still have two versions, V1 in PROD and V2 in DEV & TEST.
  

If you want to backout the change in TEST, you could migrate the object from PROD to TEST (assuming that there were no other changes in the model that would prevent this migration) an reset the status of the version to a DEV state.

Once V2 is migrated to the PROD model, you no longer have a copy of V1 in any model and could not go back to V1 through migration.

It is often useful to be able to see what the contents of previous versions was even if you do not have them in a model, and to support this, we have support for Minor Versions. Whenever an object is changed via Upload Assistant or genIE, a copy of the object is taken as a text file (PAD listing for AB/PROC, objects & properties report for ENT/WAS, source code listing for XOs) and stored in the GuardIEn database. This provides a full audit trail of each change made to an object. Whereas in a Gen model you only see the latest copy of an object and hence the detailed who/what/when of multiple changes to the same object are lost in Gen, with Minor Versions you would see all of the changes separately.

This is then a very useful resource, not only as an audit trail of changes, but also for problem solving. The ability to see the what, why, when and who (what has changed, why was it changed, when was the change made and who made it) makes diagnosing a problem much easier. With Gen, a single model can only contain a single version of an object, so if the object is changed, you lose the ability to see what it looked like the moment before the change, unless you have saved the previous version somehow (via migration, model copy, etc.).

It is possible to configure GuardIEn to manage a backup model which is maintained as part of a system update to production. The section on Backup Migration in the System Updating Steps manual describes the process that the backup migrate step uses to scope the objects that are migrated to the backup model. However because a Gen model enforces strict consistency between objects, it may not be possible for the n-1 version of two objects to co-exist in the same model, for example, if the n-1 action block uses a new attribute from the version n entity type. Hence often you may find that the current production version of an object has had to be migrated to the backup model to support the migration of another object and the backup model does not therefore contain the previous version of the object.

Gen 8.0 FP1 and Batch z/OS Libraries

A previous posting described the new Gen 8.0 feature for z/OS dynamic RI triggers and libraries and an issue related to batch support. This has been addressed in Gen 8.0 Feature Pack 1 which is currently under beta test. FP1 also introduces some a useful enhancement to host encyclopaedia construction in allowing you to specify separate on-line and batch load libraries, so that when a dynamic action block needs to be linked separately for on-line and batch, this can be done from the same business system into separate load libraries.

Continuous Integration and Gen

Continuous Integration (CI) is a practice often undertaken as part of agile development or extreme programming approaches where development team members frequently ‘integrate’ their work, often every few hours.

The requirement for code ‘integration’ most often results from the ability in most development approaches of allowing multiple developers to checkout the same source code and then rely on then ‘integrating’ their changes back into the master copy stored in a repository. It also typically results in an automated build process (and possibly also automated test) to ensure that the integrated changes are compatible with changes applied by other developers to the same or related source.

The objectives of CI are to improve the quality of software and reduce the time taken to deliver, especially by reducing or eliminating the costly and time-consuming integration tasks by ensuring that any integration issues are resolved early on and not as a massive exercise late on the development process.

Some of the technical issues that CI attempts to address are not applicable in a Gen environment. Only one person can checkout an object with modify access from a model and on upload, Gen ensures that at a basic level, the changes are consistent with the model, and so it is tempting to conclude that CI is not applicable in a Gen project.

However there are still several areas where CI concepts can be usefully applied.
The first relates to generated code and test environments. In our development environment, we generate code from the CSE into a shared development directory. In this way, every developer tests from the same code base and does not need to worry about maintaining their own private source code, object code or database. There are several other benefits of using server based code generation over local toolset generation, and perhaps this could be the subject of another post sometime…

It is important that the changes to the model, once uploaded, are correctly generated. We therefore use GuardIEn’s Upload Assistant to automatically perform the impact analysis and then generate the affected modules after each upload. In this way, the generate/build process ensures that the development code repository is kept up to date, and any errors are trapped at an early stage.

Another aspect of CI is ensuring that quality control is applied continuously. Numerous studies have shown that errors are far cheaper to fix if they are detected and fixed at an early stage in the life-cycle. We run about 25 checks on the objects changed automatically on upload using the integration between VerifIEr and the Upload Assistant. These checks detect common errors in the code (for example missing view matching or redundant code) and whilst the errors should be detected during testing, it is far easier and cheaper to correct the errors whilst the subset is still downloaded and before time has been wasted generating and testing the code.

Alternative View Mapping Technique

The recommended technique for passing data between action blocks is to use view matching on the USE statement. One disadvantage of this method is that the views need to be mapped all the way down the calling chain. For example, if we want to pass some data from AB1 to AB9, then the view(s) must be mapped on every possible USE statement:

AB1
->AB2
-->AB3
--->AB4
---->AB9
->AB5
-->AB6
--->AB7
---->AB8
------>AB9

With some very complex structures involving hundreds of possible paths through the logic, this can involve a lot of extra views being created in the intermediate action blocks in the calling chain and the potential for not mapping some of the views, thus the data is lost during the calling chain.

A technique that we have used to provide an alternative method of passing data around is to have a common action block that stores the data in an uninitialised local view.

The logic of the action block is as follows:

SAVE_DATA
IMPORTS
in action code
link my_data string (exported)
LOCALS
temp my_data string (not initialised)

IF in action code = 'P'
MOVE link my_data to temp my_data
ELSE
MOVE temp my_data to link my_data

The revised logic for the application is now:

AB1:
SET temp action code to 'P'
SET temp my_data string to 'whatever data you want to pass'
USE SAVE_DATA
WHICH IMPORTS: temp action, temp my_string

Any action block that wants the value of my_data can then use SAVE_DATA to get the value without it needing to be passed on every intermediate USE statement.

Note that this technique will only work within a single load module and cannot be used to share data across load modules unless SAVE_DATA is created as an external action block with shared memory.

In the vast majority of cases, you should still use view mapping, but there might be some cases where the above technique will allow you to easily share a small amount of temporary data between a large number of action blocks without needing to include it as data passed on all USE statements.

64 bit conversion

Gen r8 introduces the first platform to support 64 bit C code, which is HP Itanium. For the next release of our products, we will be using Gen r8 for Itanium and hence have had to port to 64 bit.

The UNIX source code generated by Gen is not specific to a particular UNIX implementation, so the same code is compiled for 32 bit on AIX and PA-RISC and 64 bit for Itanium. The difference is in the compiler options used.

One difference in the Gen r8 generated C code is that the variable used for the repeating group view 'last' flag has changed from an int to a long. In 32 bit architectures, an int and a long are both 32 bits, whereas for 64 bit, an int is still 32 bits but a long is 64 bits for the LP64 architecture used in UNIX (but still 32 bit for the LLP64 architecture used by Windows IA-64).

This means that EAB code must be modified to change an int to a long for the repeating group view variables in import and export views. You will also need to look through the EAB code to see if you have used int and long incorrectly since they are no longer the same. The same is true for pointers, which become 64 bits in both LP64 and LLP64 architectures.

Changing attribute lengths

We recently decided to increase the length of a database column (attribute) to support longer path lengths. In principle this is a very easy task:

a) Change the attribute length
b) Amend the column length in the database design
c) Use database ALTER statements to change the physical database column length
d) Re-generate the affected code

In practice however, two aspects of the change were trickier:

1) Where the attribute is referenced in external action blocks, these will need to be identified and modified.
2) Code that is dependant on the attribute length might need to be modified.

The first issue was easy to solve. We created a custom function in Object List+ that lists all external action blocks that reference the attribute in an import or export view. The resulting list was then copied to a GuardIEn Change Request and then opened in XOS. All of the affected externals could then be downloaded and modified.

The second issue was harder. We had some code that assumed the old length of the attribute (in this case 50), for example, SET text = SUBSTR(attribute,49,2) was supposed to return the last two characters of the attribute. Now I agree that this is not great code, and the attribute length could be referenced using the length function rather than 50, but it was assumed that the length would not change and the hard-coded value used instead of the length to improve performance.

To identify these occurrences, a new VerifIEr check was developed that scans for code that uses the length of an attribute as a hard-coded value. This was able to identify code that needed to be changed and can also identify any future occurrences of this style of coding that would not be tolerant of a change in attribute length.

This illustrates one of the strengths of CA Gen. Because the action diagram 'source code' is stored in a SQL database using a precise structure (as opposed to the text files used by almost any other development tool), it supports complex queries that can scan the action diagrams looking for specific coding constructs.

Of Mice and Men

Slightly off topic, but this might be of interest to older Gen developers. The Gen toolset is very 'mouse intensive' and much work needs to be done with the mouse rather than the keyboard. After 20 years of this, some of us older Gen developers are starting to feel the strain (literally), with RSI type irritations.

I found that changing to a different type of mouse was very helpful, and after trying a few out, now use a Vertical Mouse (see http://www.evoluent.com/). You may prefer a different style of mouse, and perhaps the main benefit is to change to something different?