In praise of integration

Having spent over 20 years developing our products using Gen, it is clear that one of the main benefits is the low cost of maintaining applications developed with Gen. I think that there are many reasons for this, some of which are due to inherent features of Gen and others derive from the methods and standards used by the development project. In my view, a key feature of Gen that contributes to the low cost of maintenance is the integrated nature of the analysis and design tools.

The early marketing of IEF (as Gen was called in the early days) emphasised the integrated nature of the product and IEF was called an i-CASE (integrated Computer Aided Software Engineering) tool to distinguish it from point solution CASE tools. Unfortunately many i-CASE tools were nothing of the sort and few if any came close to delivering the 100% code generation and great success of Gen. This resulted in the CASE / i-CASE market getting a bad name, through little fault of IEF.

However. having chosen the best integrated development tool, shouldn’t a Gen project maximise the benefits of that integration? The trend to only use Gen for the server and batch parts of a project concerns me. Whilst there are undoubtedly situations where Gen is not the best choice for developing the user interface, I suspect that there are others where the choice not to use Gen for the front-end has been a mistake due to the resulting increased cost of development and maintenance.

When the user interface is developed with a separate tool, the interface between the presentation layer (client) and the business logic (server) has to become much more formalised at an early stage in the life-cycle, especially when the client and server parts are developed by separate teams. Even if you are using CBD/SOA or some other development approach that advocates stable, published interfaces, there are still many situations when a rapid, iterative approach to development will benefit from having one person develop the client and its closely coupled servers at the same time and with the same tool.

The goal of 100% code generation and integrated nature of Gen means that there are boundaries to the product's capabilities. Whilst there are features that allow external code (external action blocks, OCX controls, etc.), there are still limitations on what can be accomplished with Gen. The perceived weakness of Gen for developing sophisticated user interfaces has made some Gen projects avoid Gen for the user interface or presentation layer of an application.

A few years ago, I was visiting a long standing Gen user who had used Gen very successfully to develop 3270 and batch applications. I demonstrated GuardIEn to the development manager, and then we went for lunch. He explained that they were now moving to client/server but had decided not to use Gen for the front end because they did not think that you could develop a good front end. I asked him what they were looking for, and his response was that they would like to be able to develop something that looked like GuardIEn! He did not realise that GuardIEn was a Gen developed application with the user interface created using the same Gen design tools that they had decided were inappropriate.

Now, to achieve the sophisticated look and feel of our products with Gen has not been easy. We have had to develop an add-on tool (IETeGUI) and learn how best to achieve the desired results. But is this not the case with any tool? Don’t just take the product out of the box and expect to develop a very sophisticated user interface immediately. It needs quite a bit more work than that – probably more than you would expect. It is not easy to create a great user interface with Gen, but it can and has been done, and in my view, the extra effort is more than compensated for by the significant reduction in development and maintenance effort through the use of an integrated tool with 100% code generation.

Dog Food or Champagne?

There is a saying about eating your own dog food, or the more pleasant version, drinking your own champagne. The point is that if you really believe in your own product, then you would use it yourself, and therefore I prefer the dog food analogy since you would only eat your own dog food if it was really palatable, whereas you might be prepared to drink anything that is alcoholic!

Anyway, getting back to the main point, if you are a software developer and you can use your own products, then you have a big incentive to improve them for your own benefit. This is why I was really pleased when I heard that CA would be using Gen within their development team as part of the Mainframe 2.0 initiative.

Because we develop our products with Gen, we are also able to use our own tools as well, and this positive feedback loop has resulted in many improvements and enhancements to make the ‘dog food’ as palatable as possible. An example of this is in the area of version control.

One of the most useful tools in the armoury of a developer is the ability to see what has changed in the source code. The ability to see the what, why, when and who (what has changed, why was it changed, when was the change made and who made it) makes diagnosing a problem much easier. With Gen, a single model can only contain a single version of an object, so if the object is changed, you lose the ability to see what it looked like the moment before the change, unless you have saved the previous version somehow (via migration, model copy, etc.).

Since it is impractical to save the previous version every time a change is made, often the diagnosis of a problem is made unnecessarily hard because this useful information is not available. For example, a user reports a problem in the test system that they noticed a few days ago. In the meantime, the model has been changed and you are therefore unable to see what the changes were (only that the object was last changed on a specific date/time). If you cannot reproduce the problem, you cannot then tell if the problem has been fixed, or if your test case does not properly test for the issue.

We have found the ‘minor versions’ feature of GuardIEn especially useful. This allows you to track every change made to a Gen object and see the who changed it, when it was changed and what was changed (down to properties and individual action diagram statements). When linked to a GuardIEn Change Request, you can also see why it was changed and what other objects were also affected by the same change.

I know that we would say this anyway, but we have found this capability to be invaluable in the on-going maintenance of our products.

Extending the GUI runtime

Earlier postings have discussed the benefits of a runtime, for example, simplifying the development environment and allowing developers to concentrate on the business logic. We discussed how we have extended the runtime with custom action diagram functions. However the area where we have made the most use of a customised runtime is the GUI interface.

It is no secret that the standard Gen window designer offers limited features and the resulting look and feel often leaves users underwhelmed!

The majority of our products are client/server applications with a Windows GUI presentation layer designed in Gen. A few years ago we decided that it was important to improve the usability of the products and one of the key areas that we felt needing enhancement was the GUI front-end.

Some of the areas that we felt could be improved were:

• Resizable windows and dialog boxes. These have to be designed for a minimum screen resolution (for example 800x600), but if a user has a larger monitor, it would be useful for them to be able to have larger window/dialog boxes. Two issues in a Gen application are a) if you resize a window, you then need a lot of code to reposition/resize the controls on the dialog box and b) the new size should be remembered since the user will not want to resize the dialog box each time they open it.



• List Boxes. Most windows applications utilise list boxes where the user can change column widths, re-sequence columns, sort on columns by clicking on the column heading. Often a bitmap will make the data much more readable.
  
  


• Tabs. These can reduce the complexity of an application that requries a lot of dialog boxes.

We investigated various approaches to implementing the above requirements. Most solutions involved a lot of additional code (for example to move controls around when a window is resized or to manipulate an OCX control) and would have involved a lot of effort. We also wanted to keep a common look and feel across all parts of the products and hence if we decided to enhance the products, we would have to re-code all of the windows and dialogs and not just the commonly used ones.

This is where we come back to the concept of a runtime. In our normal use of Gen, we did not have to add in a lot of window handling code – this was handled by the Gen GUI runtime. What if this runtime were extended to support our additional requirements? It transpired that a Gen list box could be ‘enhanced’ via Windows functions and still appear as a normal list box to the Gen GUI runtime. We were therefore able to extensively enhance and improve the functionality of the Gen GUI without needing to change either the action diagram code or the Gen GUI runtime.

Simple NOTE statements in the action diagram were used to provide parameters to the extended runtime and we were able to enhance the entire product line in a matter of days because the complexity was ‘hidden’ in the runtime functions and did not have to be coded by the developer.

The example below illustrates a window that utilises enhanced list boxes that can display bitmaps, it can be resized and the list boxes are resized automatically to fill the window, and various tabs are available with associated bitmaps as well.

The results of this effort are now available to other Gen sites as the IETeGUI product.

Beware hidden READ properties

We encountered a problem recently where some code worked fine on our main Oracle test system but did not work correctly on z/OS DB2.

We could not see anything wrong, so ran the code in trace using xTrace and it was then that it was noticed that there was a LIMIT 1 clause set on a READ EACH properties. This limits the rows returned to 1, but only for MS/SQL or DB2. The statement should not have had the property set, but had been copied from another READ EACH that did have the LIMIT since there was an ESCAPE after the first READ and hence it worked correctly in all databases.

This highlights the danger of using properties that affect the results set, for example, LIMIT or DISTINCT since these are not visible to the designer when viewing the action diagram.

My recommendation would be that if you decide to use these properties, add a NOTE statement indicating this above the statement. This will act as a warning if the statement is copied, and also should also indicate that the properties might need modifying.

Client/Server Versioning

Our products are designed as client/server applications where the client code resides on the user’s workstation and the server code resides on a central windows, UNIX or z/OS server.

The majority of our customers install the client code on each user’s workstation, though it is also possible to install it on a central file server. Because the changes to the workstation software are often strictly controlled and require the involvement of desktop support departments, in many sites there are restrictions on the timing and frequency that desktop software can be updated.

With a Gen client/server application, if the interface to a server changes (i.e. import/export views), then the client window manager needs to be re-generated and therefore a new version of the client software is required.

This presents two issues with the development and deployment of new releases of software.

The first issue is to prevent the client software invoking the wrong version of the server software. For example, if the server software is updated and a user attempts to access it with an older version of the client software, the software might fail (typically with a common format buffer error) or even worse, complete execution but with incorrect results.

The solution that we have adopted to the first issue is to incorporate a version number in the client software . This is passed to the server on each server call and the first thing that each server does is to verify that the client version number is compatible with the server. The version number is nearly always the same as the major release of the software and hence this ensures that the client and server software is at the same release, for example 7.7.

The second issue is to allow fixes and minor updates to the software within a release that does not require a complete refresh of the client software on all user’s workstations.

To address this requirement, we do not make any changes to the server interface (import/export views, view properties, transaction codes, etc) within a major release that would require a new client version. This means that the server code (and client code if required) can be updated to the latest service pack or with emergency fixes without needing to re-distribute or update the client code.

If there is a real need for a new client/server flow within a release, this can be accommodated with a completely new server procedure step. Updated clients can access the new p-step, but existing clients will be unaware of it and hence continue to operate unaffected. With this approach, the only consideration is that the server code will need to be deployed before the client code.

Single Logon

When launching our products (GUI client/server or local windows based ‘fat clients’) the user has to logon by providing a userid and password. These are then used to validate that the user has access to the system and optionally are also provided to the Gen client/server runtime modules for authentication at the server level, for example, to RACF on the mainframe.

Since we have multiple applications available via a program folder or accessed via the Gen toolset plug-in menu, we wanted to avoid having to ask the user to logon each time they started a new application if another one was already open.

This meant that we needed to find a way of sharing the logon credentials. This method had to be secure and reliable, so storing the values in an environment variable or file was ruled out. The solution we adopted was to use a DLL that was shared by all of the clients and the user credentials stored in the DLL’s memory with encryption.

Normally if multiple executables share the same DLL, each instance will have its own private copy of the memory, thus precluding the sharing that we were trying to achieve. The solution was to use a shared memory segment in the DLL so that the contents of the shared segment could be accessed by all executables.

The result is that once the user has logged on to one client, then all subsequent clients do not ask for the userid and password. An additional benefit of the shared DLL is that when the last client is closed, the DLL is no longer resident in memory unlike using persistent storage methods.

Memory Usage Update

A previous post described some of the techniques that we have used to reduce the memory requirements of Gen applications. One of the approaches that we have been trialling is to delay the allocation of local view storage for generated C code.

The generated C code for an action block will define a structure for the local views and the memory for these structures will be allocated for all of the action blocks linked into the load module when the load module is loaded, even if the action blocks are not called. Note that this is only strictly true for action blocks that do not have any uninitialized local views.

We have developed a source code post-processor that re-defines the local view structure so that the memory is only allocated when the action block is called and is then freed on return from the action block.

This has dramatically reduced the memory usage of the load modules, especially for co-operative servers that remain loaded by the transaction enabler.

We have been running with the modified code for a month now with no unexpected results. The benefits are a big reduction in the overall memory requirement. This allows us to improve performance by a combination of having more load modules loaded into the TE thus reducing program load time, freeing up memory that can then be allocated to the DBMS and fewer page faults.

User Defined Functions

In my last post, I described how you can extend the Gen runtime with user defined functions.

The post described how the use of user defined functions can simplify action diagrams because you can nest functions and also the input and output parameters can be specified using any attribute view of the same domain, whereas imports and exports to action blocks have to use the same entity type or workset views.

It is worth pointing out that whilst this simplifies the action diagram code, the underlying generated code will still need to ‘map’ the input and output views to additional data items that have the same length as the function’s parameters, and hence the fact that the action diagram code appears not to have this extra ‘mapping’ does not mean that there isn’t some runtime overhead associated to this data mapping.

To add a function, you have to code the logic in the same way that you would for an external action block and place the code into a library so that it is available at link time. You also have to define the function to the model, specifying the input and output parameters. These are more restrictive than for a normal action block. You are limited to one output parameter and up to 5 input parameters, and group views are not allowed. Defining the function to the model is quite a complex process, especially if you are not familiar with the Gen meta-model, so you may want to take a look at the QAT Function Manager which automates much of the tedious tasks associated with defining the function. There are also various presentations and documents produced by CA on this subject, so I do not want to repeat them here.

Once you have added your function to the model and coded the logic in C, COBOL or your other target language, you are ready to start using it in the model.

Some of the functions that we have added are:

IET_FIND. This is a replacement for the standard Gen find function which allows you to search a string using wild cards (like %) and other search operators.

IET_GET_ROW_CLICKED. This is a function that returns the clicked row given an x & y co-ordinate returned by the Gen GUI right mouse functions and a list box control. Gen only provides the x & y co-ordinates and to get the row selected, you need to adjust for the scrolling of the list box, the font height and whether the user has changed the default DPI setting in Windows.

IETeGUI functions. For IETeGUI we have developed a range of functions that allow the developer to fine-tune the GUI at execution time, for example, dynamically changing a list box column position, visibility, headings, etc. Using functions greatly simplies the coding when you want to provide detailed control over the GUI runtime.

Extending the Gen Runtime

I thought it would be worthwhile describing how we have extended the Gen runtime in the development of our tools, but first we should clarify what we mean by the ‘runtime’.

A high level language (including Gen, but also languages like C, COBOL, Java, .Net, etc,) simplifies the developer’s task by providing functions that are inherent in the programming language. Examples include printf in C, STRING in COBOL and the Gen concat function.

The actual implementation of the function by the compiler (or code generator) might be via a macro, code generation (i.e. the COBOL or C compiler generating assembler instructions to implement the statement) or via a call to a language runtime module. To the developer, it does not matter how the function is implemented. For example, the CA Gen concat function is implemented in generated COBOL code whereas in C it is implemented via a Gen runtime function call.

So what is the difference between a language’s runtime functions and those that you might procure from a 3rd party vendor or develop in-house?

From a developer’s perspective, in some languages (like C) the code to use a language function will not differ much from the code to use a 3rd party function, whereas in others (like COBOL), the in-built functions are typically referenced via language statements (i.e. STRING) whereas other functions would be used via a CALL statement. Typically a function that is invoked via simple language statements is easier to use than having to implement a separate program call to invoke the function.

In CA Gen action diagramming, the same is true. Compare the following:

SET c to numtext(substr(concat(a,b),1,5))

With the following which uses alternative routines for numtext, substr and concat.

SET map_1 TO a
SET map_2 TO b
USE my_concat
WHICH IMPORTS: map_1, map_2
WHICH EXPORTS: map_3
SET map_4 TO map_3
SET start to 1
SET end to 5
USE my_substr
WHICH IMPORTS map_4,start,end
WHICH EXPORTS map_5
SET map_6 to map_5
USE my_numtext
WHICH IMPORTS map_6
WHICH EXPORTS map_7
SET c to map_7

In the first example, one statement could achieve the desired result, whereas if we replace the use of in-built functions with external action blocks that achieve the same result, we need a lot of additional code as well as special ‘mapper’ views that are of the same type as the import and export views of the EABs.

In the above example, I have shown an extreme case where each function has its own set of import and export vies that are not shared between the functions.

Gen allows you to develop your own functions, so you could then code the above as:

SET c to my_numtext(my_substr(my_concat(a,b),1,5))

We have made use of this capability to add our own functions into the Gen model and hence simplify our code, both in terms of the number of statements needed to implement a given task, but also reducing the number of ‘mapping’ views needed.

In a subsequent posting, I will describe some of the functions that we have developed.

Windows Timers in Gen GUI

Watching paint dry is not everyone’s idea of an interesting day at work, and for certain applications this is extra tedius when you have to click a ‘refresh’ button just to see how dry that paint has become.

Various functions in our applications submit background tasks and exhibit this mindless click-to-see-if-it-has-finished scenario. After much RSI and boredom we decided it was time to implement auto-refresh and timer events in the GUI so that at least you didn’t have to click the tired old mouse so much.

To achieve this in our Gen GUI we developed an EAB which we call to register and start a timer event in Windows (WM_TIMER messages). Basically this is a Windows event interception routine which processes the timer events we are interested in, and any other events get passed on to the default window event handler (the Gen GUI & Windows runtimes).

To make something actually ‘happen’ when the WM_TIMER event is received, TIREVENT is used to trigger an event in the client PStep, and the timer event is deactivated again, as far as Windows is concerned.

The event in the PStep can then do its' thing, and if it wants to repeat the timer again (like a regular auto-refresh) it simply calls the EAB again telling it to reactivate the timer.

Pretty simple stuff in principle, but a big improvement for the poor user and their aching click-finger.