An unexpected feature of Object Migration

An unexpected feature of the CA Gen object migration utility that sometimes catches us out is when you migrate an action block and the view matching of action blocks that use it is also affected.

Consider the example where AB1 uses AB2. If you add a new import view to AB2 and view match it to an existing view in AB1:

AB1:
USE AB2
IMPORTS: temp xxx to in xxx

and then migrate just AB2 to another model, if the view temp xxx in AB1 has common ancestry between the two models, then the view match is also migrated, which in effect modifies AB1 even though this had not been selected for migrate. However AB1 does not get a modified timestamp, so it looks like AB1 has not changed even though its view matching has.

This may not necessarily create a problem, but it does sometimes cause confusion.

Gen and null columns

A recent posting on the Duick forum regarding NULL column support lead to a discussion on the Gen qualifier IS EQUIVALENT TO and a potential misinterpretation of the way that this works.

As a bid of background information, it is important to understand how a nullable column containing a NULL value behaves. Consider a table with a nullable column and these rows:

Id opt_column
1 ' ' (column has a value of spaces)
2 NULL(column is NULL
3 'X' (column has a value of X)

If you want a value of NULL to be equivalent to SPACES, and you want to read rows that have a space or null in the opt_column, then if your SQL was:

SELECT * FROM table WHERE opt_column = ' ';

would return just row 1 but

SELECT * FROM table WHERE opt_column = ' ' OR opt_column IS NULL;

would return rows 1 and 2

If you want to read rows that do not have ‘X’:

SELECT * FROM table WHERE opt_column != 'X';

would return just row 1 but

SELECT * FROM table WHERE opt_column != 'X' OR opt_column IS NULL;

would return rows 1 and 2

Once you understand the need for the IS NULL or IS NOT NULL qualifier in the SQL, you can write the READ qualifiers in the action diagram code.

The confusion arises over the use of the IS EQUIVALENT TO clause since it is likely that this does not work the way you expect!

For example, if an optional column has no value, then I think of SPACES & NULL as the same, so you would code:

READ table WHERE opt_column = SPACES OR opt_column IS NULL

However the statement READ table WHERE opt_column IS EQUIVALENT TO SPACES

gives the following SQL which is not the same:

SELECT opt_column FROM table
WHERE (opt_column = ' ' AND opt_column IS NOT NULL)

This means that if the column is NULL it will not return a row, which is the opposite of what I think you want.

IS EQUIVALENT TO is OK. For example, if you want a row with a specific value:

READ table WHERE opt_column = ‘X’ AND opt_column IS NOT NULL

Is the same as READ table WHERE opt_column IS EQUIVALENT TO ‘X’

which gives the following SQL:

SELECT opt_column FROM table
WHERE (opt_column = 'X' AND opt_column IS NOT NULL)

IS NOT EQUIVALENT also gives the desired result for a specific value but not for SPACES:

READ table WHERE opt_column NOT = ‘X’ OR opt_column IS NULL

and READ table WHERE opt_column IS NOT EQUIVALENT TO ‘X’ gives:

SELECT opt_column FROM table
WHERE (opt_column <> 'X' OR opt_column IS NULL)

But if you want a row where the column is not spaces, you would code:

READ table WHERE opt_column NOT = SPACES AND opt_column IS NOT NULL

but READ table WHERE opt_column IS NOT EQUIVALENT TO SPACES gives:

SELECT opt_column FROM table
WHERE (opt_column <> ' ' OR opt_column IS NULL)

In summary, it is best not to use EQUIVALENT with SPACES unless you want the behaviour that the generated code gives you. You would also need to be careful with a view that might have a value of spaces.

Documenting Changes

Whether you are developing software that is commercially available or for internal use, users value a clear description of the enhancements and fixes introduced in a new release or service pack. This is especially important when there are changes in behaviour or actions that need to be taken by the users to take advantage of new features.

A customer recently complemented us on the quality of our release notes and asked whether we generated them from a database.?

Unfortunately there wasn’t a magic solution. We cut and paste the descriptions of each change into a Word document and then generate the PDF file from that. That part is simple though. The harder part is to ensure that each change is documented properly and is not accidentally omitted from the release notes. We therefore document the changes using the following process:

1) Each change must be documented in a form that will make sense to the end user, explaining the business reason for the enhancement or requirement for a fix. The documentation is in the long description of the Change Request (CR) in GuardIEn.

2) Each CR should address a single problem or enhancement. You should avoid CRs that span multiple requirements (the worst cases being a single CR that has all changes in it or a CR for changes made by a developer that is not linked to the actual requirements.

3) If a new requirement is found whilst changing some code, a new CR should be created for it and the temptation to ‘hide’ the new requirement within the scope of the existing CR avoided.

4) Once the CR has been completed and tested, the description should be reviewed for accuracy and any changes in behaviour noted.

5) The Release Notes should be updated with the CR and user documentation reviewed and updated as necessary. We have a separate state in the CR life-cycle to indicate that the documentation has been updated.

Almost like having a new machine

We recently upgraded our anti-virus software to the latest 2010 release and it was immediately noticeable how much slower our machines were, especially our heavily used CSE machine. We also found the desktops to be much slower, and so the new release was de-installed and the older (2009) release re-installed. It still took up to 25% of the CPU though, and so we decided to try some alternatives. After a bit of research, we selected one of the other leading products to trial. Both desktops and the CSE run much faster and it is like having a new machine and so the upgrade can be delayed for a while!

In praise of integration

Having spent over 20 years developing our products using Gen, it is clear that one of the main benefits is the low cost of maintaining applications developed with Gen. I think that there are many reasons for this, some of which are due to inherent features of Gen and others derive from the methods and standards used by the development project. In my view, a key feature of Gen that contributes to the low cost of maintenance is the integrated nature of the analysis and design tools.

The early marketing of IEF (as Gen was called in the early days) emphasised the integrated nature of the product and IEF was called an i-CASE (integrated Computer Aided Software Engineering) tool to distinguish it from point solution CASE tools. Unfortunately many i-CASE tools were nothing of the sort and few if any came close to delivering the 100% code generation and great success of Gen. This resulted in the CASE / i-CASE market getting a bad name, through little fault of IEF.

However. having chosen the best integrated development tool, shouldn’t a Gen project maximise the benefits of that integration? The trend to only use Gen for the server and batch parts of a project concerns me. Whilst there are undoubtedly situations where Gen is not the best choice for developing the user interface, I suspect that there are others where the choice not to use Gen for the front-end has been a mistake due to the resulting increased cost of development and maintenance.

When the user interface is developed with a separate tool, the interface between the presentation layer (client) and the business logic (server) has to become much more formalised at an early stage in the life-cycle, especially when the client and server parts are developed by separate teams. Even if you are using CBD/SOA or some other development approach that advocates stable, published interfaces, there are still many situations when a rapid, iterative approach to development will benefit from having one person develop the client and its closely coupled servers at the same time and with the same tool.

The goal of 100% code generation and integrated nature of Gen means that there are boundaries to the product's capabilities. Whilst there are features that allow external code (external action blocks, OCX controls, etc.), there are still limitations on what can be accomplished with Gen. The perceived weakness of Gen for developing sophisticated user interfaces has made some Gen projects avoid Gen for the user interface or presentation layer of an application.

A few years ago, I was visiting a long standing Gen user who had used Gen very successfully to develop 3270 and batch applications. I demonstrated GuardIEn to the development manager, and then we went for lunch. He explained that they were now moving to client/server but had decided not to use Gen for the front end because they did not think that you could develop a good front end. I asked him what they were looking for, and his response was that they would like to be able to develop something that looked like GuardIEn! He did not realise that GuardIEn was a Gen developed application with the user interface created using the same Gen design tools that they had decided were inappropriate.

Now, to achieve the sophisticated look and feel of our products with Gen has not been easy. We have had to develop an add-on tool (IETeGUI) and learn how best to achieve the desired results. But is this not the case with any tool? Don’t just take the product out of the box and expect to develop a very sophisticated user interface immediately. It needs quite a bit more work than that – probably more than you would expect. It is not easy to create a great user interface with Gen, but it can and has been done, and in my view, the extra effort is more than compensated for by the significant reduction in development and maintenance effort through the use of an integrated tool with 100% code generation.

Dog Food or Champagne?

There is a saying about eating your own dog food, or the more pleasant version, drinking your own champagne. The point is that if you really believe in your own product, then you would use it yourself, and therefore I prefer the dog food analogy since you would only eat your own dog food if it was really palatable, whereas you might be prepared to drink anything that is alcoholic!

Anyway, getting back to the main point, if you are a software developer and you can use your own products, then you have a big incentive to improve them for your own benefit. This is why I was really pleased when I heard that CA would be using Gen within their development team as part of the Mainframe 2.0 initiative.

Because we develop our products with Gen, we are also able to use our own tools as well, and this positive feedback loop has resulted in many improvements and enhancements to make the ‘dog food’ as palatable as possible. An example of this is in the area of version control.

One of the most useful tools in the armoury of a developer is the ability to see what has changed in the source code. The ability to see the what, why, when and who (what has changed, why was it changed, when was the change made and who made it) makes diagnosing a problem much easier. With Gen, a single model can only contain a single version of an object, so if the object is changed, you lose the ability to see what it looked like the moment before the change, unless you have saved the previous version somehow (via migration, model copy, etc.).

Since it is impractical to save the previous version every time a change is made, often the diagnosis of a problem is made unnecessarily hard because this useful information is not available. For example, a user reports a problem in the test system that they noticed a few days ago. In the meantime, the model has been changed and you are therefore unable to see what the changes were (only that the object was last changed on a specific date/time). If you cannot reproduce the problem, you cannot then tell if the problem has been fixed, or if your test case does not properly test for the issue.

We have found the ‘minor versions’ feature of GuardIEn especially useful. This allows you to track every change made to a Gen object and see the who changed it, when it was changed and what was changed (down to properties and individual action diagram statements). When linked to a GuardIEn Change Request, you can also see why it was changed and what other objects were also affected by the same change.

I know that we would say this anyway, but we have found this capability to be invaluable in the on-going maintenance of our products.

Extending the GUI runtime

Earlier postings have discussed the benefits of a runtime, for example, simplifying the development environment and allowing developers to concentrate on the business logic. We discussed how we have extended the runtime with custom action diagram functions. However the area where we have made the most use of a customised runtime is the GUI interface.

It is no secret that the standard Gen window designer offers limited features and the resulting look and feel often leaves users underwhelmed!

The majority of our products are client/server applications with a Windows GUI presentation layer designed in Gen. A few years ago we decided that it was important to improve the usability of the products and one of the key areas that we felt needing enhancement was the GUI front-end.

Some of the areas that we felt could be improved were:

• Resizable windows and dialog boxes. These have to be designed for a minimum screen resolution (for example 800x600), but if a user has a larger monitor, it would be useful for them to be able to have larger window/dialog boxes. Two issues in a Gen application are a) if you resize a window, you then need a lot of code to reposition/resize the controls on the dialog box and b) the new size should be remembered since the user will not want to resize the dialog box each time they open it.



• List Boxes. Most windows applications utilise list boxes where the user can change column widths, re-sequence columns, sort on columns by clicking on the column heading. Often a bitmap will make the data much more readable.
  
  


• Tabs. These can reduce the complexity of an application that requries a lot of dialog boxes.

We investigated various approaches to implementing the above requirements. Most solutions involved a lot of additional code (for example to move controls around when a window is resized or to manipulate an OCX control) and would have involved a lot of effort. We also wanted to keep a common look and feel across all parts of the products and hence if we decided to enhance the products, we would have to re-code all of the windows and dialogs and not just the commonly used ones.

This is where we come back to the concept of a runtime. In our normal use of Gen, we did not have to add in a lot of window handling code – this was handled by the Gen GUI runtime. What if this runtime were extended to support our additional requirements? It transpired that a Gen list box could be ‘enhanced’ via Windows functions and still appear as a normal list box to the Gen GUI runtime. We were therefore able to extensively enhance and improve the functionality of the Gen GUI without needing to change either the action diagram code or the Gen GUI runtime.

Simple NOTE statements in the action diagram were used to provide parameters to the extended runtime and we were able to enhance the entire product line in a matter of days because the complexity was ‘hidden’ in the runtime functions and did not have to be coded by the developer.

The example below illustrates a window that utilises enhanced list boxes that can display bitmaps, it can be resized and the list boxes are resized automatically to fill the window, and various tabs are available with associated bitmaps as well.

The results of this effort are now available to other Gen sites as the IETeGUI product.

Beware hidden READ properties

We encountered a problem recently where some code worked fine on our main Oracle test system but did not work correctly on z/OS DB2.

We could not see anything wrong, so ran the code in trace using xTrace and it was then that it was noticed that there was a LIMIT 1 clause set on a READ EACH properties. This limits the rows returned to 1, but only for MS/SQL or DB2. The statement should not have had the property set, but had been copied from another READ EACH that did have the LIMIT since there was an ESCAPE after the first READ and hence it worked correctly in all databases.

This highlights the danger of using properties that affect the results set, for example, LIMIT or DISTINCT since these are not visible to the designer when viewing the action diagram.

My recommendation would be that if you decide to use these properties, add a NOTE statement indicating this above the statement. This will act as a warning if the statement is copied, and also should also indicate that the properties might need modifying.

Client/Server Versioning

Our products are designed as client/server applications where the client code resides on the user’s workstation and the server code resides on a central windows, UNIX or z/OS server.

The majority of our customers install the client code on each user’s workstation, though it is also possible to install it on a central file server. Because the changes to the workstation software are often strictly controlled and require the involvement of desktop support departments, in many sites there are restrictions on the timing and frequency that desktop software can be updated.

With a Gen client/server application, if the interface to a server changes (i.e. import/export views), then the client window manager needs to be re-generated and therefore a new version of the client software is required.

This presents two issues with the development and deployment of new releases of software.

The first issue is to prevent the client software invoking the wrong version of the server software. For example, if the server software is updated and a user attempts to access it with an older version of the client software, the software might fail (typically with a common format buffer error) or even worse, complete execution but with incorrect results.

The solution that we have adopted to the first issue is to incorporate a version number in the client software . This is passed to the server on each server call and the first thing that each server does is to verify that the client version number is compatible with the server. The version number is nearly always the same as the major release of the software and hence this ensures that the client and server software is at the same release, for example 7.7.

The second issue is to allow fixes and minor updates to the software within a release that does not require a complete refresh of the client software on all user’s workstations.

To address this requirement, we do not make any changes to the server interface (import/export views, view properties, transaction codes, etc) within a major release that would require a new client version. This means that the server code (and client code if required) can be updated to the latest service pack or with emergency fixes without needing to re-distribute or update the client code.

If there is a real need for a new client/server flow within a release, this can be accommodated with a completely new server procedure step. Updated clients can access the new p-step, but existing clients will be unaware of it and hence continue to operate unaffected. With this approach, the only consideration is that the server code will need to be deployed before the client code.

Single Logon

When launching our products (GUI client/server or local windows based ‘fat clients’) the user has to logon by providing a userid and password. These are then used to validate that the user has access to the system and optionally are also provided to the Gen client/server runtime modules for authentication at the server level, for example, to RACF on the mainframe.

Since we have multiple applications available via a program folder or accessed via the Gen toolset plug-in menu, we wanted to avoid having to ask the user to logon each time they started a new application if another one was already open.

This meant that we needed to find a way of sharing the logon credentials. This method had to be secure and reliable, so storing the values in an environment variable or file was ruled out. The solution we adopted was to use a DLL that was shared by all of the clients and the user credentials stored in the DLL’s memory with encryption.

Normally if multiple executables share the same DLL, each instance will have its own private copy of the memory, thus precluding the sharing that we were trying to achieve. The solution was to use a shared memory segment in the DLL so that the contents of the shared segment could be accessed by all executables.

The result is that once the user has logged on to one client, then all subsequent clients do not ask for the userid and password. An additional benefit of the shared DLL is that when the last client is closed, the DLL is no longer resident in memory unlike using persistent storage methods.