z/OS Catalog fault finding and fixing

This page describes how to use raw IDCAMS to investigate and fix catalog problems. It is usually easier to use a maintenance product for this. The shortcuts below will take you to specific points on the page.

Listing catalog information

The ICF catalog structure is complex, so if you have a problem, the first thing to work out is exactly which component is broken. Typically you'll be passed a problem ticket from someone who cannot access his data. Start by querying the catalog status for an affected file, by using the listcat command. The easiest way is to pick up a file list from ISPF option 3.4, then type the following line command against a file.

LISTCAT ENT(/) ALL

Here's a tip which you probably know already. You'll find yourself using the listcat command all the time. Shortcut it by sticking the following clist into a command library in your logon procedure. Type TSO LISTA to find an appropriate library. Add the clist as member name LC.

PROC 1 MEMBER
LISTC ENT(&MEMBER) ALL
EXIT CODE(0)

Now when you want to list the catalog entry for a dataset, all you need to do is type LC against it. The LISTCAT output tells you; which catalog the dataset entry is in, which volume the catalog thinks the dataset is on, and SMS class information. You get a lot more detail for VSAM datasets.

Checking the physical condition of a Catalog

If you are getting a lot of problems in the same catalog, then it is possible you have a physical error on the catalog dataset (it is just a VSAM KSDS). Physical errors often cause the catalog backups to fail too. If you use LISTCAT ENT(/) ALL against a catalog, it will tell you which master catalog it is in, which volume it is on, and a list of all the aliases which point to it. To see the full VSAM information, you need to use listcat ent(/) catalog(/) all

To check the physical status of the catalog, run the following IDCAMS statements in batch

EXAMINE NAME( -
CATALOG.ICFUSER.VOL001 - ) -
INDEXTEST DATATEST -
ERRORLIMIT(50)

and hopefully you see the output

IDC01700I INDEXTEST BEGINS
IDC01724I INDEXTEST COMPLETE - NO ERRORS DETECTED
IDC01701I DATATEST BEGINS
IDC01709I DATATEST COMPLETE - NO ERRORS DETECTED

If the examine reports problems, then you might have to recover the catalog, as described on the next page. However, before you start to recover, try an IDCAMS verify first, as it might just be that the file was not closed properly. The syntax is

VERIFY DATASET(catalog.name)

However, if you have a catalog maintenance product onsite, it will have functions to allow you to fix physical catalog errors. The maintenance product section has more details.

Checking the logical Catalog condition

The following IDCAMS command will check that the dataset entries in a catalog are complete. For example, it will check that VSAM truenames are associated with a cluster record.

DIAGNOSE ICFCATALOG -
INDATASET(CATALOG.ICFUSER.VOL001) -
LIST

and you should see
IDC01360I THE FOLLOWING ENTRIES HAD NO ERRORS:
followed by a list of all the entries

If your diagnose runs clean, but you still get catalog errors, try refreshing the catalog buffers using the console command

F CATALOG,CLOSE(catalog.name)


The following command will check out the logical structure of a VVDS

//DD1 DD DISP=SHR,DSN=SYS1.VVDS.VVOL001
//SYSIN DD *
DIAGNOSE VVDS -
INFILE(DD1) -
LIST

All the previous two commands do is check out that a single catalog entity is correct. What you need, is to ensure all the catalog components are consistent.

//DD1 DD DISP=SHR,DSN=SYS1.VVDS.VVOL001
//SYSIN DD *
DIAGNOSE ICFCATALOG -
INDATASET(Catalog.name) -
COMPAREDD(DD1)

This will check that the entries in a catalog are consistent with those in one VVDS. It does not check out the entries in all the other VVDS datasets. To find out which VVDSs are referenced by a catalog, use the command

listcat level(sys1.vvds) catalog(CATALOG.ICFUSER.TSG)

Two errors are possible. If the VVDS contains datasets, but there are no catalog entries for them, create them by using

DEFINE CLUSTER (NAME(SYS1.VVDS.Vvolser) VOLUME(volser)-
NONINDEXED RECATALOG) -
CATALOG(catalog.name)

If the Catalog contains entries, but they are not in the VVDS and don't physically exist, use

DELETE SYS1.VVDS.Vvolser NOSCRATCH CATALOG(catalog.name)

The next command will validate a VVDS against a catalog

//DD1 DD DISP=SHR,DSN=SYS1.VVDS.VVOL001
//SYSIN DD *
DIAGNOSE VVDS -
INFILE(DD1) -

COMPAREDS(Catalog.name)

If you end up with a missing VSAM Volume Record (VVR) for one component of a VSAM file, then do a delete / noscratch at the cluster level first to clean things up. That leaves all the components uncatalogued. Then do a define without the recatalog parameter, specifying the correct volumes and allocated space for the file you are fixing. So in the IDCAMS statement below, change the dataset name, volumes and space as appropriate.

DEFINE CLUSTER -
(NAME(OPROD.TEST)) -
DATA (NAME(OPROD.TEST.DATA) VOL(V3003F) TRK(3,1) ) -
INDEX (NAME(OPROD.TEST.INDEX) VOL(V30038) TRK(1,1) )

IBM removed support for VSAM parameters IMBED and REPLICATE some time ago. They must be removed from all catalogs before you install z/OS 1.8 or they will cause abends.

The number of processing strings is quite critical to catalog performance as they control the number of possible parallel read requests. The default value is 2, but this is too low. Consider changing your strings 5 or 7, depending on how busy it is. You will also need the correct number of data and index buffers. 5 data buffers and 7 index buffers are about right for 5 strings. If you define more buffers than this you are probably just wasting storage.

Extended Catalog Sharing can improve performance between LPARs as this puts the VVDS information into the coupling facility. This was problematical at first but seems to be stable now.

You may also get response time problems in a shared environment if you are converting hardware reserves to dataset enqueues with CA-MIM. Consider using GRS in a STAR configuration instead.