THe HRS2 Technical Manual - Population CouncilThe Project organizes and provides access to€ HRS2 files Object Oriented Design in HRS2 ... Modifying the WHRS database Consistency

The HRS Technical Manual Page 1 of 2

file://F:\hrsmanuals\HRS2TECH.html 9/15/00

The HRS2 Technical ManualVersion 1.0

Overview of Longitudinal Studies and the HRS2

Longitudinal Household Studies The HRS2 Special Studies

HRS2: A System to Manage Longitudinal Population Data

2.1 The Functional Components of the HRS2 Software 2.2 HRS2 Data Model 2.3 Internal Logic and Operations of the HRS2 System

The Organization of the HRS2 Program

The Project organizes and provides access to HRS2 files Object Oriented Design in HRS2

Making Changes to the HRS2

Form Attributes and methods used by HRS2 Control Attributes and methods used by HRS2 This, Thisform, and other Object Oriented Names HRS2 Data Environments and Views Classes contained in the HRS class library GHRS Global Attributes and Methods used by the HRS HRS2 Forms and the HRSformcontroller object How to run Portions of the Program Interactively Generating a Database Schema

Consistency Checks for HRS2 Fields

Writing Simple Consistency Checks Writing Consistency Checks for Multiple Fields in the same file Writing consistency checks that look across multiple files (databases)

How to Add New Variables to the HRS2

Making Changes to the Field Register Adding a new field to a table Adding a new field to a data entry form Changing the validation routines to report inconsistencies

The HRS Technical Manual Page 2 of 2

file://F:\hrsmanuals\HRS2TECH.html 9/15/00

How to create a new data entry form

Bringing Legacy data into the HRS2

Modifying the WHRS database Consistency Checks on Legacy data Consistency Checks put in Forms

How to Link Special Studies into the HRS2

How to Make Changes to the Core of the HRS2

Frequently Asked Questions (FAQs)

Appendix A (Structure of HRS2 database tables)

Appendix B (Specification files in HRS2.PRJ)

Appendix C (Specification files for Special Study Programs)

Acknowledgment

Parts of this document have been adapted from manuals prepared for the Navrongo Health Research Center (NHRC), Ghana. This document is part of research is supported by grants from the Thrasher Foundation, the Mellon Foundation, and The Population Council, New York.

Bruce MacLeod, Phd.James F. Phillips, Phd.

The Population Council

One Dag Hammarskjold PlazaNew York, NY 10017

USAJanuary, 1998

Overview Page 1 of 3

file://F:\hrsmanuals\Overview.html 9/15/00

Overview of Longitudinal Studies and the HRS2

Longitudinal Household Studies

For many health studies, households need to be studied over a period of time. Cross-sectional demographic data collection methods are inappropriate for many health research needs. Demographic events of interest, such as neonatal, post-natal, child and maternal mortality are sufficiently rare as to require precise information on the events of interest and the population at risk. Moreover, cross-sectional survey techniques, so widely used in fertility research, are ill-suited to mortality studies because the proximate determinants of mortality - morbidity, nutritional status, lactation behavior, etc. are either intervening events or longitudinal processes that can only be effectively studied in concomitant event history analyses. Furthermore, health service interventions improving child health are time conditional, in that the impact of services derives from the timing of interventions relative to the season of adversity and specific episodes of illness.

In addition to studying a population over time, household characteristics and member relationships can clarify the causes and consequences of morbidity and mortality in developing countries. Strong observed correlation's, between social and economic status and health status attest to the value of information on household member relationships, customs, and behaviors in research on the determinants of survival. The household unit of analysis is also critical to research evaluating health interventions, since health service effectiveness is determined as much by household social and behavioral factors as by the efficacy of medical technology.

Despite the theoretical and methodological appeal of longitudinal household studies, however, such studies typically fail to achieve their objectives. Reasons for failure vary, but prominent among them are the data management problems of longitudinal household studies. Recording relationships is a complex undertaking when pursued in longitudinal studies because events of interest must be recorded over time and linked with other characteristics of individuals. Furthermore, the analysis of illness and mortality rates must take these changing population characteristics into account. A final complication, common to many cohort studies, is the tendency of investigators to encumber research with more complexity than is required. Awareness of the complexity of the social and behavioral determinants of health status often invites the collection of large amounts of data that are difficult to compile, manage, and analyze.

Despite data management difficulties that can mar research or delay progress,



longitudinal household studies retain their methodological appeal. Quite apart from the theoretical justification for such designs, interest in longitudinal studies has been stimulated by methodological innovations for the regression analysis of life tables and concomitant event data. Relatively little progress has been registered, however, in improving capacities to compile and manage the requisite information for this research.

The HRS2

The HRS2 is a software system that

l maintains a consistent record of significant demographic events that occur to a population in a fixed geographic region,

l generates up to date registration books which are used by the field workers,

l and computes basic demographic rates (births/age, deaths, ...).

These features are built into a framework that is built to support change. Investigators add to and sometimes modify this core set of data and software to tailor the HRS2 to their particular projects. Data fields can be added, logic and consistency checks can be written, and the screen layout can be interactively specified by the investigator.

Programmers who modify and extend the HRS2 software system deal with very short pieces of code which are attached to a field in a table or a control object in a data entry screen. This provides an object oriented software development approach that is well suited for software development. An alternative software development approach is to start writing code -- lots of it. From our experience in Bangladesh and Indonesia it takes many thousands of lines of computer code to correctly model the intricacies of the problem. The process is time consuming, the software maintenance costs are high, and well trained computer programmers are needed. But the rewards for solving the problem are also high -- for example, the Bangladesh software system, called the Sample Registration System (SRS), was the computational foundation for hundreds of research papers concerning issues of maternal and child health, mortality, and family planning strategies.

Attempts by other organizations to adapt the Bangladesh SRS program to their circumstances were largely unsuccessful though. It was simply too complex to remold the many thousands of lines of computer code to the special requirements of a new site -- no amount of documentation, manuals and tutorials could overcome the technical barriers. The HRS2 was designed, to a large extent, to facilitate the transfer of software technology for the longitudinal monitoring of households.

Special Studies

The HRS2 provides an information infrastructure for special projects. The HRS2 databases can be used to extract relevant samples of members. Data



collected from these special studies can be linked to the longitudinal histories of household members.

As an example of a special study, we consider a research center in Navrongo, Ghana where the outcomes from changing the orientation of health services from a predominantly clinical approach to a community-based strategy are being documented and analyzed. The primary research source of the Navrongo Health Research Center (NHRC), is a district-wide Navrongo Demographic Surveillance System (NDSS). This system records vital demographic events and insures that the demographic impact of health services can be the subject of systematic trials. The NDSS has been built from the HRS2.

The NHRC conducted a cross-sectional study on women of child-bearing age (i.e. women between the ages of 15 and 45 was conducted by the NHRC. Before the study, the NDSS was used extensively to sample a sub population of women in the 15 - 45 age group. The NDSS divides the NHRC study area into four distinct areas (or zones) and the sample was to get an equal proportion of women from all four zones. Using the NDSS baseline databases, women household members within the study area were retrieved with all vital information such as IDs., names, and date of birth.

Field workers were trained and taught how to administer a questionnaire with relevant issues on women of child-bearing age. Together with the list of sampled household members, the field workers were able to visit each sampled woman in her compound. Special data entry screens were designed to help put answers of the questionnaire in a computer. With computerized data, it was possible to link this one-time study using the IDs. of the sampled women to the on-going longitudinal demographic system of the NDSS.

In later sections of this document, we address the problem of developing data systems for special studies.

hrs2 Page 1 of 12

file://F:\hrsmanuals\system.html 9/15/00

HRS2

A System to Manage Longitudinal Population Data

The HRS2 structures data and maintains logical integrity on the following basic elements of a household cohort:

l All households have defined members. Rules unambiguously exclude non-members.

l All households have a single head, and members relate to one another and to the head in definable ways.

l Members have names, dates of birth, and other characteristics that do not change.

l Dynamic events include pregnancies, births, deaths, in and out migration, and marital status change.

l Longitudinal events occur to individuals at risk (i.e. active members) and must follow simple logical relationships. Observation of a household defines the person-days of risk that accrue to all members.

l Dynamic events can change household membership or relationships according to fixed rules.

The HRS2 thus defines the structure of households, demographic events to members, and the population at risk.

Constructing an accurate database of longitudinal household information require some care. Everyday relationships tend to become complex and unwieldy when arrayed as a logical system of longitudinal population data. Furthermore, portraying even simple relationships requires rigorous standards to avoid error. For a member of a household to die, for example, he or she must obviously first be alive and be present as a household member. Boring as such logic may seem, lapses in the integrity of data management can generate deaths to individuals who are not logically members of the risk set, births to non-existent mothers, or migrations among the deceased. Even small numbers of such offending events can mar the utility of data for serious analysis. Errors generated at one stage tend to cause many additional errors in later stages; this compounding effect can quickly cause the database to become completely out of step with the population of study. Longitudinal household research thus requires defining rigorous and unambiguous standards for data management.

The HRS2 contains software that allows households and household members to be added to or deleted from the database, reports to be generated, and existing data to be edited. Investigators add to and sometimes modify this core set of data and software to tailor the HRS2 to their particular projects. Data fields can be added, logic and consistency checks written, and the screen

hrs2 Page 2 of 12


layout (for data entry) can be interactively specified by the investigator. HRS2 data and code specification, and the ability to modify and add to the specification, all combine to give the user the flexibility to develop a site-specific household registration program.

The Functional Components of the HRS2 Software

Viewing the HRS2 software from a user's perspective provides an overview of the HRS2 program structure. Screens of the software system as they appear to a data entry clerk are provided first. Afterwards, some of the data entry screens that a programmer would use to extend the software are provided and explained.

When a user first logs onto the HRS2, the following main menu screen appears :

The options in the main menu of the HRS2 are:

l Data entry: Allows for the entry, deletion, and editing of the baseline and longitudinal data. Baseline household information includes information about the household location, individuals within the household, relationships between individuals, and familial social groups. Longitudinal information includes basic information related to births, deaths, migrations in and out of the study area, and marriages.

l Validation: Subsets of households and members can be checked for

hrs2 Page 3 of 12


logical consistency in the validation option.

l Reports and Output: Calculates and displays the demographic rates and life tables. Age specific and overall rates are computed.

l Visit Register : Used to print the household record book. The household record book is used by the field workers to record information during household interviews.

l Utilities: An option that is primarily used by the system administrator. It includes capabilities for adding new user IDs, setting interview round information, and generating reconcilation reports to help track down unreported pregnancy outcomes and unmatched internal migrants.

Collectively, these functions would form a part of every application generated from the HRS2.

A great deal of the user interaction with the HRS2 occurs in the data entry screens. Two screens are fundamental to the data entry process: the baseline screen which allows the data entry from the initial enumeration, and the update screen which allows the entry of longitudinal information. The data entry window for baseline information is presented in the following figure :

hrs2 Page 4 of 12


Much of the information displayed in this screen can be adjusted to suit project specific needs. For example, the labels and many of the field values can be changed. Also, while the HRS2 requires that all locations and individuals have unique IDs, the format of the ID (character, numeric, ...) can be changed from project to project. The HRS2 requires that the gender of individuals be input to the system, but again, the format of this information ((M,F), (1,2), ...) can be changed. More details are provided in the user documentation.

After entering the initial enumeration of locations, individuals, and relationships, the HRS2 is used to generated field books for entering demographic information collected during subsequent visits to the locations. After every visit round, the field worker will hand the books to the computer center and the changes in the demographic status of the household can be entered using the update screen :

A data entry clerk will enter the information at the top of the screen (location ID, ...). Once this information is provided, the grid of individuals currently resident at the location is displayed. If an event occurred to an individual, say a birth to Ajua Adugbire, then the user would scroll down to the relevant individual and click on the button representing the event (in our example, the Pregnancy Outcome button would need to be pressed). Clicking on buttons causes an event specific entry screen to appear that collects additional information about the event (an example of an event screen is provided

hrs2 Page 5 of 12


below).

The above screen can be adapted to accommodate new types of "events" by adding additional event buttons. For example, suppose that information about malaria fever episodes occurring to children under five years old is required. A "Malaria fever" button could be added and checks could be put in place to restrict entries to the appropriate subset of individuals (in this example, under five children).

The event forms collect additional information about the particular event. One of the more complex events is a pregnancy outcome. The event screen is given below:

Information can be added to this core data about a birth. For example, a birth attendant field could be added to the information at the top of the screen and a birth weight field could be added to the child level fields in the grid.

The other event forms are similar in structure (although in most cases, less complicated) than the Pregnancy outcome form.

hrs2 Page 6 of 12


OUTPUT

Once data has been collected over a time interval, the HRS2 provides capabilities for generating key demographic rates, such as fertility, mortality, and migration. In addition the HRS2 can generate a population distribution by age and life tables. The various rates can be calculated for a user specified time interval and for different geographic subsets of the population. An example of the output from a fertility rate calculation is given below :

Life table output is another option and the following screen illustrates the output :

2.2 HRS2 Data Model

The HRS2 Data model reflects the logical organization of tables and relations that underlay the

hrs2 Page 7 of 12


HRS2 software. The following diagram provides an overview of the relational structure.

The three different types of objects maintain static information about locations, individuals, and social groups (which are usually the family, but could be other groupings of individuals). In the RDM, a record in the individual file does not necessarily mean that the individual is present in the study. An individual is associated with a physical location in the study at a given point in time through a resident episode.

Episodes record an interval of residency, membership to a social group, or a relationship between two individuals. All episodes contain an individual ID, the starting and ending dates, starting and ending observation IDs, as well as the events that initiated the start and end of the episode. A resident episode records the stay of an individual at a particular physical location. A membership episode records the membership of an individual to a particular social group. Finally, a relationship episode records a relationship, such as marriage, between two individuals.

hrs2 Page 8 of 12


Events indicate a transition in the state of an individual. Birth, death, in migration, and out migration events effect changes in the composition of the individuals in a study. Events, which are not intended to record the start and end of a particular episode, are known as "point events". The pregnancy outcome event records the termination of a pregnancy. Pregnancy outcomes that result in live births will also have one birth record for every child born. The status point event is meant to record the change of an individual's status. This could include the pregnancy status of a mother or the nutritional status of a child.

The observation entity stores the information that a particular physical location has been observed at a given point in time. Information on the person making the observation and optional information such as the census round can also be stored as part of this entity. The observation entity is linked to all the events recorded during the observation.

INTERNAL LOGIC AND OPERATIONS OF THE HRS2 SYSTEM

Typically, a research site will need to extend the core HRS specification to include new systems of variables on household attributes, individual characteristics, or events. An understanding of the internal logic of the HRS2 is needed before such changes can be made.

The HRS2 is built from the form (data entry screen), menu, and database builders of the Microsoft Foxpro System (currently developed in Version 5.0). These builders encourage and facilitate a modular, object-oriented software development approach. In the HRS2, most of the objects represent variables from the HRS core tables and these objects can appear on the data entry, reporting, and rate generation forms. Small pieces of code, called code snippets, are "attached" to these objects. Some code snippets control when data can be entered for a variable, others enforce data consistency rules.

Collectively, the database, form, and menu specifications along with their associated code snippets, define the HRS core. When changes need to be made to the core, a programmer locates the database table, menu, or form object where changes need to be made and then works with the code snippets attached to the object. Since there are only a few code snippets attached to an object and these code snippets are usually short, the process of modifying and extending the HRS is significantly easier than changing code in a thousand line program.

This method of modifying and extending software is very different from the process used only a few years ago. The old method of software development required a programmer to manage both the control logic of a program (for example, the sequence in which fields in a data entry screen are visited) as well as the semantic logic (for example, consistency checks). When changes needed to be made, the relevant code section had to be located in the many thousands of lines of code, changes made, and then the programmer hoped that those changes did not cause problems with the control code in other

hrs2 Page 9 of 12


portions of the program. Only experienced programmers could work with large software systems written in this way.

The new object oriented and visual style of programming is much easier for both beginning and experienced programmers. Point and click specifications combined with a reasonable default behavior allow the beginning programmer to develop adequate software systems. In addition, this new form of software development allows an experienced programmer to develop objects that can be dropped into an application. These objects can then be used be used by beginning programmers to develop new applications. The HRS2 contains many "pre-cooked" objects that should prove useful in demographic surveillance systems.

To better appreciate this new programming landscape, a few examples are considered in more detail. The intent is not to teach programming, but to illustrate the new style of interacting with the software.

The Foxpro form builder allows objects in a form specification to be visually manipulated. When a programmer wants to change the form layout or variable labels, the appropriate object is selected and then moved, edited, or deleted. It may be necessary to modify the form specification for a number of reasons. Reasons for changing a form include adding a variable, changing data prompts, changing logical constraints, or adding controls governing when data can be entered. As an example, the update form specification appears to the programmer as follows :

hrs2 Page 10 of 12


Obviously, the specification and the data entry form seen by users are visually very similar. If there is a problem with a field, say the Location Number, then a programmer can mouse click on the location field and bring up the list of properties associated with that field and make any corrections. Another not so obvious point concerns the control buttons at the bottom of the screen. These buttons contain a substantial amount of logic to control user interaction with the form. Many hours of system development went into making this set of buttons and the associated properties applicable to a large number of data entry screens. A beginning programmer, who needs to create a new data entry form, need not understand any of this logic. They can simply paste the control buttons onto their new data entry form and inherit all the embedded logic. This is an extremely powerful means of code reuse. If the reuseable objects are sufficiently powerful and useful, then programmers need only insert the right objects in the right places.

Asking for the properties of any of the objects embedded in the form will reveal more of the details of that object's behavior and characteristics. For example, a few of the properties associated with the date control on the update form are shown below :

hrs2 Page 11 of 12


Each control object has tabs to reflect the various categories of attributes and behaviors. The Data tab allows the programmer to associate the object with a particular field in a table. Methods allows the programmer to attach computer code to particular events that occur during the running of the program. In the above example, the When code snippet returns true if a user can enter data into the field, and false otherwise (in this particular case, the code in the lower box restricts the entry of data so that this field cannot be edited after the data has been entered). The Layout attributes affect the look of the object on the form and the Other tab contains a few miscellaneous attributes.

Consistency logic can also be associated with the objects embedded in a form. But often a better place to put this logic is in the database table itself. Consistency logic can be associated with a field in a table or with the entire row in the table. If data is inserted into the table, then the consistency checks are applied to the data before it is inserted; any consistency errors and the data cannot be inserted. This capability is one of the most significant and powerful features for the HRS2 programmer. It allows changes to be made to some of the HRS2 consistency checks (for example, changing the allowable codes for relationship to household head) as well as adding new project specific logic to HRS2 fields or new fields. The following figure shows some of the fields for the individual table as well as more detailed information, including a consistency rule, for the gender field.

hrs2 Page 12 of 12


We anticipate that most of the consistency logic for an application can reside in the table definition. This means that data entry screens that use these fields will only need to specify when the data should be entered as well as the position and layout of the data field.

organization Page 1 of 4

file://F:\hrsmanuals\organization.html 9/15/00

The Organization of the HRS2 Program

This section describes the general organization of the HRS2 program. It is assumed that you are familiar with the FoxPro System tools; if not then a good reference is Microsoft Visual FoxPro 3, the Pros Talk , published by Microsoft Press (ISBN 1-57231-23305) (it is ok that it is titled 3.0 rather than 5.0 - the small differences between 3.0 and 5.0 do not affect the development in HRS2).

The HRS2 program is distributed with all its source code. It runs either as an executable file (i.e. WHRS.EXE running directly from the DOS command prompt) or as a FoxPro application file (i.e. WHRS.APP running within the FoxPro environment). To run the HRS2, one needs either the executable file or the application file and HRS2 database files (or tables). Appendix A contains a detailed description of all the HRS2 database files.

The Project organizes and provides access to HRS2 files

To modify and extend the HRS2, you need to access the files making up the software. The HRS2 software is built from the form, database and menu builders of Visual FoxPro 5.0 development environment. These tools encourage and facilitate a modular, object-oriented software development approach. The project file is the primary entry point for accessing any of the source files that make up HRS2. After opening the whrs.pjx project file (menu item file, then sub-option open then choose whrs.pjx from the subdirectory that it is stored in), the following screen will appear :



The tabs at the top of the project window indicate the different categories of files. The Data category includes all tables, views, and stored procedures (functions and procedures that go with the database). The Docs category includes all screen forms and reports. The classes category includes the hrs class library which contains some of the key structural components of the HSR2 system. Code contains program files and other contains the menu files. Double-clicking on any of the files will bring up the appropriate design tool. For example, double clicking on one of the tables will table design tool. So the project is the conduit to accessing the HRS2 files.

Object Oriented Design in HRS2

Object oriented software combined with visual development tools make a very powerful combination. The HRS2 takes advantage of this technology in a number of ways. First, most components on HRS2 screens are built in an object oriented manner. This allows the developer to make changes to underlying components and the changes will ripple through the system. Next, the HRS2 has two non-visual objects that are key to the operation of the program. The application object contains code that, among other things, brings up menus and screens and shuts the system down. The GHRS object contains global variables and function that influence the operation of the program. These variables include the data format, values for yes and no, and a function to build an ID for an individual. This section describe the HRS2 object oriented design in more detail.

The visual classes are best understood by looking at the design of a data entry form. As an example, consider the process of building one of the event forms - say the out migration form. There is a lot of similarity between the event forms; event forms have a control bar on the bottom, they have identical error handing capabilities, they ask for field worker code, and they calculate event id and status of data values. These commonalities can be captured in a visual form class (called eventform in the hrs class library) that looks like (if you want to see this in FoxPro, choose the classes tab in the whrs project, then click on the entry called eventform in the hrs class library. Later we discuss the process of designing a new event form):



In addition to the obvious visual components, this form contains data fields and functions (called methods) and that influence the operation of this form. To see these hidden attributes, you can right mouse button click on an empty portion of the form and choose the Properties submenu option. After widening the size of the properties window, something like the following appears:

Anything in bold is changed from the default value. In this figure, the form has initialization code (in the init event method) and has a font size of 11. Suppose that you changed the font size to 12 in the event form. All events forms would then display text in a size 12 font.

The process of building the out migration form starts with an event form (in object oriented parlance, the outmigration form inherits the attributes and behaviors of the event form). To verify that this is the case, you can open the outmigration form (under the Docs tab of the whrs project), and look at the



properties. Under the other tab of the property sheet, notice that the class attribute is eventform.

All the controls in the outmigration form are also based on classes contain in the hrs library. These controls can be made accessible to the programmer designing a form by making the form control toobar visible (view menu, toolbars submenu option) and then adding hrs controls by choosing the view classes icon add adding the hrs library. From here, the controls can be dragged and dropped into a form. Some of the ID fields on the form, such as the individual ID, allow a programmer to drag and drop a control that will allow the user to type in an ID (and only accept legal IDs) or pull down a list of IDs. Another control, hrsfield, is used for the entry of most values including numeric, character and date information. It contains logic that allows the user to abandon entry, checks for validity from the database tables as well as a number of other features. You can design screens without using these controls but all the capabilities that exist in hrsfield would have to be replicated. To see which control a screen is based on, you can right mouse button click on that control and choose properties. The class attribute indicates the class the control is based on.

Of the two non-visual classes, an HRS2 programmer will most likely work with the GHRS class most. The GHRS class is contained in the hrs library. It serves as a mechanism to maintain most configurable attributes and functions in one place. Here is where you can specify the length of an ID field or how to build an ID field. To access these settings, examine the properties of the class.

changes Page 1 of 7

file://F:\hrsmanuals\changes.html 9/15/00

Making Changes to the HRS2

This section considers general issues which are relevant to the process of changing the HRS2 software system.

While there are a number of different types of changes that can be made to the HRS2, the same overall sequence is used. First, a complete backup of the entire system is made in some other directory (or machine). All changes are made to the backup and the backup is only used after testing and debugging is completed. It is often useful to bunch together a collection of changes and release a new "version" with its own release number. Backups should be made of all versions. The process of making changes starts with a programmer using the project file to locate the menu, form, or table object where changes are needed.. The appropriate object is selected and modified. When all the changes are complete the project rebuilt.

Locating the place where changes are to be made is usually very easy. It is the control or form where changes are needed. In an object oriented system, the attributes (data) and behaviors (functions) are usually close by. Although sometimes the computer code that needs to be changed is located in the class a control or form is built from. In this case, the programmer needs to go to the hrs library.

This chapter contains an assortment of topics that are important to know before a programmer delves into making changes to the HRS2. It begins with a description of the form and control properties that are used by the HRS2. For each control or form, there are a very large number of attributes or methods that could changed (at least 100). To minimize the potential for confusion, the HRS2 restrict changes to a limited number of attributes and methods. Next, this chapter includes a description of the attributes and methods of the global class GHRS. This class allows the programmer to making easy changes that can affect the overall operation of the program. After this, a few sections discuss object orient programming and naming conventions. Finally, this chapter concludes with a description of the pre-packaged classes that are provided with the HRS2 system.

Form Attributes and methods used by HRS2

The form attributes and methods used by the HRS2 are listed below with a brief explanation. See the FoxPro documentation for more details. You can find all the attributes and methods listed in the properties of the form. Some of the attributes and methods are defined as part of the form. Others have been defined in the process of designing the HRS2. We indicate this latter group by putting (HRS2) in parenthesis.

l Buffer Mode : indicates whether, by default, the data tables and views used by the form will be buffered (kept in a temporary storage area while changes are being made)

changes Page 2 of 7


l Caption : Specifies the text in an objects caption l Error event method : code that is processed when an error occurs l Init event method : code that is processed when the form first starts. l refresh event method : can put code in that will be processed every time

the form displays itself. l addmode method (HRS2) : a method that is used by programmers to

determine whether data is being edited or added. Some fields should not be changed when editing and this function returns True only if data is being added.

l deletestatus method (HRS2) : the deletion status of the underlying record -- method return .T. if record is deleted

l displayerrormessage method (HRS) : Displays an error message and allows a user to cancel out of the add or edit

Control Attributes and methods used by HRS2

The data entry control attributes and methods used by the HRS2 are listed below with a brief explanation. See the FoxPro documentation for more details. You can find all the attributes and methods listed in the properties of the control. Some of the attributes and methods are defined as part of the control. Others have been defined in the process of designing the HRS2. We indicate this latter group by putting (HRS2) in parenthesis.

l Comment : Text that describes the control l ControlSource : very important field that specifies a table or view field

that the control is associated with. l Format : Controls the format of the incoming data (for example !A

indicates only upper case letters). l MaxLength : maximum number of characters or digits that can entered for

the field value l Name : Specifies a name for the control that can be used in program

code. The HRS2, by convention, specifies all control names starting with the letter o following by the field name

l ReadOnly : Indicates whether the control is read-only or not. l Refresh event method : can put code in that will be processed every time

the control displays itself. This is especially useful for fields that are not tied to the active table or view (the table or view that gets changed when the controls next, prev, … are pressed)

l StatusBarText : specifies the text to display in the status bar when the control has focus.

l Valid event method : Should NOT be used as this will override the HRS2 error handling behavior

l When event method : code to indicate when data can be entered for this control.

l Value : used in program code to indicate the value (usually typed in by the user)

l CheckFieldRule method (HRS2) : A method that checks the field rule in the database table. This method should only be changed when that behavior needs to be altered.

l CheckValidity method (HRS2) : This is where you can place all your

changes Page 3 of 7


consistency checking code. l displayerrormessage method (HRS2) : Displays an error message and

then allows the user to cancel out of the add/edit of the the record.

This, Thisform, and other Object Oriented Names

Object Oriented program introduces a bit more complex naming convention than occurs in more traditional programming languages such as Pascal or Fortran. This section presents small programming code snippets to help explain object oriented naming conventions :

First, we consider the code that is often present in the when method of a control that cannot be edited. This code is as follows :

if thisform.addmode() return .T. endif return .F.

Thisform refers to the form that the control has been inserted into. The dot following thisform indicates that an attribute or method of thisform will be reference following the dot. Addmode is method (defined by HRS2) that returns True is the user is adding data and not editing it.

Consistency checking code can be associated with the database table field or placed in the checkvalidity method of a control. The following code segment is associated with a control that will only accept Y or N

if (this.value <> "Y") and (this.value <> "N") thisform.displayerrormessage("Invalid value for Internal -- Enter Y or N") return .F. endif return .T.

This.value refers to the value entered by the use (this. is a reference to the control object). If the user has not entered a Y or N, then False is returned, otherwise True is returned.

Finally, some objects can act as containers the method of specifying components of a container is to specify the container followed by a dot and then the component name. So, for instance, suppose a data control is inside a form and it is named oDate. The value of this field can be referenced by another control with the following name :

thisform.oDate.value

changes Page 4 of 7


HRS2 Data Environments and Views

The tables and views that are associated with a form are given as part of the data environment. The data environment can be obtained by clicking the right mouse button on an empty region of the form and choosing the data environment.

Almost every form in the HRS2 has tables and Views that are associated with it. Typically, the main data for a form is a view. Views allow data to be entered into a record that is written to a table once all the data has been entered . Therefore, since all the data in a record is inserted into the table at once, strong consistency checks which may look at multiple fields can be written and associated with the database table. In addition, views allow data for records from more than one table to be entered in a single record.

Classes contained in the HRS class library

Changing and modifying the HRS2 requires some familiarity with the classes that serve as the building blocks for the HRS2 system. The HRS2 has been purposefully constructed in a way that facilitates the insertion and deletion of modular building blocks. To see a list of these classes, choose the classes tab on the project, click on the + symbol next to hrs, and then double click on any of the entries. The following list describes the key entries in the HRS library :

l entry form :generic hrs2 data entry form. Has hrsformcontroller object, error handling, and HRS2 initialization.

l eventform : for capturing events (point in time data) l episodeform : for capturing episodes (interval data) l objectform. : for capturing baseline table information, such as location,

individual, …

MORE to come…..

GHRS Global Attributes and Methods used by the HRS

The GHRS class serves as a container for the global attributes and methods of the program. The oGHRS object is an instance (the only instance) of the GHRS class and it is available to all portions of the program. In this subsection, we list the attribute, its default value and a brief explanation of the attribute. Many of the attributes have the same name as their value. While this may seem redundant, it allows the program to easily change the value of the attribute without having to go through all the HRS2 source code and make those changes. So, for example, if the value for a Male was 1, then a programmer can change the value of GHRS.male from 'M' to 1 and the program will run correctly with the new male type value. For methods, we just provide

changes Page 5 of 7


the explanation.

l yes = "Y" *-- Yes response l no = "N" *-- No response l male = "M" *-- Male l female = "F" *-- Female l abt = "ABT" *-- Abortion-- used in Pregnancy outcome type l bir = "BIR" *-- Birth -- used in Member entry type and Pregnancy outcome type l chd = "CHD" *-- Three character code to indicate a child relationship l div = "DIV" *-- A marital status type indicating Divorce l dth = "DTH" *-- Value signifies Death (used in the Mortality Event form) l enu = "ENU" *-- Entry Type for Enumeration l ent = "ENT" *-- Migration in value (ENTRANCE) l ext = "EXT" *-- Migration out value (EXIT) l fam = "FAM" *-- event when an individual is associated with a family social group l gdp = "GDP" *-- Three character code to indicate a grandparent relationship l mar = "MAR" *-- A marital status type indicating Marriage l mig = "MIG" *-- Code for a migration -- used to indicate the entry type of an in-migrant l mis = "MIS" *-- Miscarriage -- used in Pregnancy outcome type l mrt = "MRT" *-- A type indicating a Marital status relationship l obs = "OBS" *-- An observation event type -- used in the Pregnancy observation form l one = "ONE" *-- Relationship to head value indicates the head of household l oth = "OTH" *-- Three character code to indicate an other relationship l par = "PAR" *-- Three character code to indicate a parent relationship l pregobsv = "OBS" *-- Pregnancy Observation l rec = "REC" *-- A marital status type indicating Reconciliation l sep = "SEP" *-- A marital status type indicating separation l unk = "UNK" *-- Character code to indicate an unknown relationship l wid = "WID" *-- A marital status type indicating Widowing l wif = "WIF" *-- Character code to indicate a wife relationship l lenobserveid = 9 *-- The number of characters in the observation ID l lenindividid = 10 *-- The number of characters in the individual ID l lensocialgpid = 9 *-- The number of characters in the socialgpid l lenlocationid = 7 *-- Number of characters in the Location ID l notapplicableid = "NA" *-- The value of the ID that represents not applicable l unknownid = "UNK" *-- The value of the ID that represents an unknown ID value l emptyoid = "" *-- Empty observation ID l emptytype = "" *-- Empty Type l emptydate = { / / } *-- Empty date l emptyid = "" *-- Empty ID value l queryid = "Q" *-- The value of the ID that represents a query to be resolved by further

research l minmarage = 12 *-- Minimum Age of Marriage l minpreginterval = 90 *-- The minimum interval in days between a birth and another

Pregnancy l earliestenudate={01/01/1993} *-- The earliest date in which a member can be enumerated

in the system l or field work l pending = "P" *-- Pending Validity Status for a record l valid = "V" *-- Valid Validity Status for a record

changes Page 6 of 7


l warning = "W" *-- Warning Validity Status for a record l dateformat = "AMERICAN" *-- The format of the date field in the data entry screens l databasename = "whrs" *-- The name of the database l relaxidrules = .F. *-- A boolean flag to indicate whether id checking rules would be

temporarily relaxed. Used BaseLine, inmigration, and pregoutcome data entry. This value can be used by the programmer.

GHRS Methods

l build_eventid *-- Construct an event ID -- currently formed by incrementing # in setup file l build_individid *-- Build an Individual ID from a location and a member number l build_episodeid *-- Build an episode ID l build_locationid *-- Build a location ID from a region and a house number l build_observeid *-- Build an Observation ID from a location and a round l build_socialgpid *-- Build a Social Group ID l setbrowsemode *-- Sets the Menu entries so the User can browse through the Data l setmenumode *-- Turns the menu back on

HRS2 Forms and the HRSformcontroller object

The HRS2 class library has four types of forms that facilitate data entry : entry form, eventform, episodeform, and objectform. These forms serve as starting points for constructing a new data entry form. To see the attributes of these form, choose the classes tab on the project, click on the + symbol next to hrs, and then double click on any of the entries with the above names.

Each of these forms has a reference to an object named hrsformcontroller. This object contains some fairly complex software to navigate, save, edit and delete records. It should not be necessary for a user to get inside any of this code (although the more technically inclined may be inserted - in which case they can edit the buttonset object in the hrs class library). Some of the methods in hrsformcontroller are designed to help build new data entry screens. These methods and their function are listed below :

afterdelete : method which is called after a deletion to the current record is made afterrecall : method which is called after a recall to the current record is made aftersuccessfulupdate : called after a successful update to the tables is made. Useful for making changes to related tables. beforeadd: method that is called before Data is added. Useful for initializing any controls that are not attached to the active table or view beforeupdate : method call that is made just before the call to update the tables -- useful for setting calculated values and checking if all important values have been assigned. Function returns a value of .F. if update cannot be made, .T. otherwise transactioncode : method call made in the middle of the transaction that saves all changes made to buffered tables. Useful for adding changes to related tables. If transaction fails, then changes in this method will not be updated.

This information should prove useful for those projects who are designing new forms. The creation of new forms is the subject of one of the later chapters in this document.

changes Page 7 of 7


How to run Portions of the Program Interactively

Most of the forms can be run without starting from the main program (whrs which calls the switchboard form). This can be very useful when testing and debug changes to a form. Before a form can be run though, you must run an initialization program called starter located in the Code section of the project.

If a form is not working correctly, there are a few tricks you can use to help find the problem. At any place in the code you can place a suspend statement. This will suspend execution of the program. At this point you can bring up the debugger by choosing Tools menu, then Debugger or typing debug in the command window. The debugger will allow you to execute the program in a step by step manner or to place execution breakpoints which serve to suspend the program at a particular line. Another useful technique is to look at the Data Session window and examine the opened tables and views.

Generating a Database Schema

Microsoft provides a utility to generate a database schema which defines and documents your table someplace other than the table itself. The schema can be used to generate the database.

To run this program type

do \Visual FoxPro\Tools\GENDBC\GENDBC with <Schema output File Name>

<Schema output file name> is the name of the file you want GENDBC to create. The output from this program is very useful for documentation and backup purposes.

consistency Page 1 of 7

file://F:\hrsmanuals\consistency.html 9/15/00

Consistency Checks for HRS2 FieldsThe HRS2 has many built-in consistency checks that restrict the data to logical values. Without a high level of consistency checking, the data tends to fall apart under the weight of errors. A project may very well want to change or tighten some of the checks provided by the HRS2. This section address the issue of adding consistency checks to the HRS2.

Consistency checking restricts the values of fields in a table. Some consistency checks restrict the value of a field to some subset - such as M and F for gender. We will refer to these as field level rules. Other consistency checks check that two or more values in a record are consistent - such as birth_date and mother's age. We will refer to these as row level rules. Finally, consistency checks that restrict the value of a field based on values found in other tables will be referred to as crosstable rules. An important type of crosstable consistency checks restrict the values of an ID field to a value found in some other file. We will refer to these checks as relational integrity rules. The HRS2 is designed to support all of the above different forms of consistency checks.

Generally, these consistency rules can be placed in three different locations:

l Database table : when you double click on a database table, a list of the fields comes up. Associated with each field is a Field validation rule and validation text. You can type in an expression and that expression can reference functions. Many consistency functions reside in the stored procedures section of the database. Stored procedures are carried with the database, so these checks maintain integrity no matter how the data is entered. If you click on the table tab, then you will notice that there is a slot to specify row level checks. These checks are made once all the data for a record is assembled. Placing consistency rules in a table is usually the best way to enforce consistency. Most of the HRS2 rules are placed in the table.

l Form : Associated with each HRS control is a method called checkvalidity (for other non-hrs controls there is a valid method - don't use this method on HRS controls though as it will mess up the error handling). You can place code in the method which returns True (.T.) if the field is valid, false otherwise. Any manner of check can be placed in here. These validity checks will be enforced when a user enters data using the form. If the user does not use the form and edits the table directly, then the consistency checks will not be performed.

In addition to the checkvalidity method, HRS controls will test the validity status of a field by using the rule that is associated with the field in a database table. Therefore, HRS2 form control data is consistent with the field checks in a database.

l Validation Checks : You can place consistency checks in the Validation



module of the HRS2. These checks will only be performed when someone runs the validation module.

The HRS2 will allow consistency checks to be placed in more than one location.

In this section, we look at three different examples: a simple consistency check, consistency checks involving multiple fields within the same database file; and consistency checks involving multiple database files.

Writing Simple Consistency Checks

The main task for consistency checks lies in the writing of computer code. This code is then attached to a validation rule in a database table, or put in the checkvalidity method of a form control, or finally it could be placed in the one of the validation check methods associated with the Validation form.

One simple form of consistency checking is to restrict the value value of a field to some subset of values. Fox example, the gender of an individual is needed. To restrict the value of this field to "M" or "F", the following code can be placed in the validation rule slot of the field gender in the table individual:

(gender = "M") or (gender = "F")

In addition, text indicating the type of error should be placed in the validation message box. Anyone who modifies the database at any time will need to put in data that conforms to this rule. This includes editing raw data files, putting data in through the data entry program, etc. If a data entry screen has an HRS2 control associated with this field, then the database field rule is checked immediately after the user enters in a value for the field. Non-HRS controls do the check at different times - not always when you want it performed.

Alternatively, we could put this consistency check in the checkvalidity method of an HRS2 control. In general terms, a simple form of consistency checks in the checkvalidity method is as follows:

If (this.value = Valid Code) return .T. endif return .F.

where this.value is the value typed into the control by the user and Valid Code is the valid values that are acceptable at a site. Logical operators such as AND and OR are used to evaluate more than one condition at a time, for cases where there are more than one valid code.

For most fields, including the gender field, the hrsfield control is used as it collects character, dates, and numeric data. The code to put in the



checkvalidity method of the hrsfield control associated with the gender field is :

if (this.value = "Y") or (this.value = "N") return .T. endif return .F.

The ANDs, ORs, and NOTs can be strung together, So, for example, if we assign minus one (-1) as a query value for the number of sleeping rooms in a household , disallow an empty field and accept numbers between 0 and 99 then the checkvalidity method for the sleeping rooms field, can be written as shown below with lines beginning with ** as comments:

** Only allow numbers betwee 0 and 99 If not empty(this.value) and (this.value >= 0) and ( this.value < 100) return .T. endif return .F.

Finally, if we wanted to put this check in the Validation module of the HRS2, we would put the following code segment in the onememberchecks method of the Validation form :

if (gender <> "Y") and (gender <> "N") thisform.CheckOutput(search_mem,"99","Individual : value for gender field <> M or F ") fflg = thisform.resetflag(fflg,"F") endif

The methods checkform and resetflag are methods of the Validation module that facilitate error reporting. Checkoutput writes out the ID (in this case search_mem), the error number, and a text explanation. Resetflag resets the status flag to the most serious error status found in the record so far. Another simple form of validation can be enforced by deciding when a user can put data in a control. The When method of a control determines whether data entry or data modifications are to be permitted on the value of the control. If a false value is returned, users cannot enter data in the control. Many of the when methods in HRS2 controls are the same - they will not allow data to be edited, only entered in append mode. The code for the When Clause is as follows:

If thisform.addmode() return .T. endif return .F.

5.2 Writing Consistency Checks for Multiple



Fields in the same file

Writing some consistency checks may demand having to reference information already collected either within the same database file or across multiple database files. In this section we look at the case of writing consistency checks for fields within the same database file.

Two examples are considered. In the first example, consistency is established between two fields, the start date and the end date, in an episode file. In the second example, consistency is established between the records in the same file.

For the first example, a consistency check is needed to restrict the end date of an episode to be greater than the begin date of an episode. The ResidencyRowChecks stored procedure is part of the database and it serves as the record row checker for the table Residency (see the record validation slot under the tab TABLE for the Residency table). The consistency rule should be placed in this procedure - associating it with a field would restrict the order of data entry. The code to put in the ResidencyRowChecks is as follows :

m.sdate = residency.sdate m.edate = residency.edate if not empty(m.edate) and (m.sdate > m.edate) whrsadderror("Starting date is later than the ending date ") m.validitystatus = .F. endif return m.validitystatus

The procedure WHRSadderror is a stored procedure that collects all errors for another stored procedure whrsdisplayerror to display them. A quick way to get to any stored procedure is to double click on any of the stored procedures, then right mouse button click - a list of methods will come up, chose Residencyrowchecks and close the dialog.

To put the same consistency check in a data entry form, a check in the edate (end date) control needs to check that a value for sdate has been entered and then compare the two fields. The code in the checkvalidity method of the edate control would look something like the following :

m.sdate = thisform.osdate.value if empty(m.sdate) this.displayerrormessage(" cannot enter an ending date without putting in starting date first ") return -3 endif

if (this.value < m.sdate) this.displayerrormessage(" Ending date is less than starting date ")



return .F. endif

Thisform.osdate.value gets the value of sdate from the form control associated with sdate. A return -3 statement moves the focus of control three fields back (the location of the sdate control).

To put this rule into the validation module would require some work for, at the moment, we do not look into event or episode files to check the consistency of an individual. In the near future, this should be added.

In the second example, we check for consistency in the father IDs that are entered in the individual record. The basic idea is to check that we have either a legal ID value or one of the possible values for an ID field that is unknown. If there is a legal ID value, then the gender of the father ID is checked. The field level rule that is placed in fatherid of the table Individual is as follows :

otherid(fatherid).OR.legalmaleid(fatherid)

Both otherid and legalmaleid are stored procedures associated with the database. While it is not necessary to know how legalmaleid works, it may be instructive to look inside the stored procedure. A portion of the code in this procedure follows :

m.savealias = alias() select individid,gender from Individual where individid == pindividid into cursor ccheckList m.individid = ccheckList.individid m.gender = cchecklist.gender m.reccount = reccount('ccheckList')

IF USED("ccheckList") USE IN ccheckList ENDIF

if not empty(m.savealias) select (m.savealias) endif

if (m.reccount = 1) and (m.individid == pindividid) and (m.gender = "M") return .T. endif

return .F.

m.savealias is a local variable that holds the value of the currently open table (as given by alias()). The select statement is an SQL command that extracts a portion of data from a table. The select command is used throughout the HRS2. An understanding of the select command is essential for making modifications to the HRS2. The technical manual does not address the issuing



of writing SQL statements as this is done in a number of places. The select command creates a new table (cCheckList) and makes this table active. After extracting the data from the table, it is closed and the m.savealias table is made active. The above code is very similar to a large number of consistency checks in HRS2.

The code in the checkvalidity method of a HRS2 control associated with a fatherid field would look like the following :

if otherid(fatherid).OR.legalmaleid(fatherid) return .T. endif return .F.

Note that in baseline data entry, the consistency check will look in the individual file as well as in the collection of records that are currently being entered. This gets a bit more complicated and is not central to this presentation. See the code for more details.

Putting this code in the validation module is a straightforward extension of the concepts presented above so it is not covered in any more detail.

5.3 Writing consistency checks that look across multiple files (databases)

In certain cases consistency rules may be need to check against date data residing in different database files. This subsection describes how to do this.

All events and episodes have at least one observation ID associated with the record. The observation ID allows someone to link into the observation table to determine the field visit date, the location, and the field worker who found the event or episode. A referential integrity check is needed to only legal observation IDs.

The field level rule associated with the observeid field in any one of the event or episode tables is as follows :

observeidfound(observeid)

Again, we make use of a stored procedure in the database - in this case, the observeidfound procedure. To gain more familarity with the SQL select command, we look inside the observeidfound procedure :

local m.individid, m.reccount, m.savealias * * Certain entry operations (such as Baseline) enter in a collection of records at one time. * Since the IDs are constructed in a consistent manner and it is very difficult



* to check for consistency among partially committed records, we allow an escape *

if (type('oGHRS') == 'O') and oGHRS.relaxidrules return .T. endif

m.savealias = alias() select observeid from Observation where observeid == pobserveid into cursor ccheckList m.observeid = ccheckList.observeid m.reccount = reccount('ccheckList')

IF USED("ccheckList") USE IN ccheckList ENDIF

if not empty(m.savealias) select (m.savealias) endif

if (m.reccount = 1) and (m.observeid == pobserveid) return .T. endif

return .F.

Notice the similarity with the legalmaleid stored procedure described above. This code includes a few more lines that were actually present in the legalmaleid method. These lines check whether the GHRS objects has been created and, if it has been created, is the relaxidrules variable set to True. The oGHRS.Relaxidrules variable is used when a collection of records are entered and those records point to one another for ID values. Since the HRS2 does not force a sequence for the tables to be saved in and the SQL select command only extracts from saved records, we cannot do complete ID checking in some data entry forms. The baseline data entry form is one example - fortunately, in this form, the IDs are constructed in a way that the new records are consistent.

Add New Variables Page 1 of 5

file://F:\hrsmanuals\AddNewVars.html 9/15/00

How to Add New Variables to the HRS2

The HRS2 database contains the essential data for demographic analysis and rate computations. Often, however, there will be the need to add to this minimal database. In this section we look at a typical example of collecting information related to the number of sleeping rooms at a location - this information is currently not included in the HRS2.

The steps to add a new variable to the HRS2 are:

1. Make changes to the field register to allow for field entries of data specific to the study.

2. Add a new field to the appropriate database table. 3. Add new field to all data entry forms that reference the altered table. 4. Write consistency checks 5. Optional : Change the validation routines to report inconsistencies.

Making Changes to the Field Register

The HRS2 field register is divided into four main sections - information related to field workers, locations, individuals in the location, and events occurring to individuals. New information can thus be inserted into the register depending on its relevance to the four sections. The files regis80 and regis132 contain the specifications for the 80 column and 132 column registers. The FoxPro Report generator can be used to modify the existing register format by creating enough spaces for the valid values of the new variable and a description of the variable.

Adding a new field to a table

Deciding which table gets the new field is not always an easy decision. A complete explanation would bring us into the theory of database design and database normalization. To keep things short, a few guidelines are suggested. First, the field should go to the table that it is most closely related to. Secondly, avoid repeating a field in many records when the field could go to another table in fewer records. For example, a new field that describes the ethnic group could go to the social group or an individual. The decision would be based on whether all members of the social group have the same ethnic value. If not, then ethnic must go to the individual table.

Sometimes, a field changes through time. In this case, you must plan to have a table which records the changes in the field through time. Consider the problem of modeling malaria fevers in children over five. Suppose that the field worker should record any children with a fever from the time of the last



visit. In this simple case, we could use the status observation table (indvstatus) by adding a new type value (MAL for example). The changes would be minor. But suppose that the field worker needed to collect additional information about the malarial fevers - such as temperature, number of incidents, … In this case, a new event table must be formed. Constructing new event tables is the subject of a later section.

We consider the simpler case where a field must be added to an existing table. Suppose we wish to add "number of sleeping rooms in a location" as new information to the HRS2. Designate the new field name as SLEEPRM. Since sleeprm is related to locations, this variable will be added to the location table. We modify the structure of the location by adding the following field:

The steps to do this are as follows :

1) Open up the WHRS project, choose the data tab, and double click on the table which will contain the new variable. The table editor appears.

2) Insert the field along with its name, associated type, and size. If you can specify consistency rules for the field, then do so in the rule option.

3) All views which reference the altered table need to be changed.

The new field is now present in the table. The next step is to allow this field to be entered in the data entry forms.

Adding a new field to a data entry form

After creating the new field in the database table, the next task will be that of modifying all data entry forms used either for entering the new variable or for displaying the new variable. The following provides the details of making alterations to one data entry form :

1) Choose the form tab on the WHRS project

2) Open up the form(s) that reference the table that you just altered (by double clicking on the form name)

Field name: sleeprmField type: numericField width: 2Field decimal: 0

Field rule sleeprm > 0 and sleeprm <100



Sometimes a tables' data will appear on more than one entry forms. For example, entry for the location table appears in the location form as well as the baseline form. In the case of the individual table data, it is referenced in four forms : individual, baseline, inmigration, and birth.

3) Form design screen is open. Now we need to make the HRS2 controls available. These controls incorporate many of the features in the HRS2 such as escape from data entry, field level rule checking, … The results will be unpredictable at best if you do not use HRS2 controls. To make these controls available, make sure the Forms control toolbar is visible (if not then choose the View menu option, followed by toolbars suboption). Click on the View classes Icon in the forms control toolbar (looks like a collection of books), and choose the hrs library in the classes subdirectory of your project. The controls should appear as icons.

4) You need to associate your new field with a HRS control. Most of the time, the hrsfield control is the best. In any case, choose the appropriate control and insert it into the form.

5) Associate the control with the new field in the table by clicking the right mouse button on the new the control. Choose the properties submenu option. Locate the control source property, click on it and then use the pull down list to choose the new field you inserted.

Also, by convention, I change the name of the field to be equal to o + <name of field>. This convention helps clarify that the screen entity is really an object that contains attributes (data fields) and behaviors (methods). Attributes and behaviors control the way that this screen object behaves.

6) Insert a label for the new field by choosing the HRS label control and clicking that control into the form. Choose the properties of the label to change the Caption (which is the text that appears on the form)

7) Set the tab order of the new control. Tab order indicates the order that controls are visited when a user is entering in data. Choose the View option and the tab order suboption. Click on the controls in the order that they will be visited in.

If you want to put consistency checks or control when data can be entered, then you may need to make alterations to the when and checkvalidity methods of the control you just inserted. See the discussion in the previous chapter for more details on how this is done.

Changing the validation routines to report inconsistencies

Validation is a main menu option which allows for batch consistency checks to be applied to data after some of it has been entered. Typically, the checks to put in this file are those that cannot be checked at data entry (e.g. number of people in a household > 0) or those variables which contain query values. Most data consistency checking can be done at the database level or the form level. To some extent, this module is a leftover from the MS-DOS version which did not allow database table rules. For the sake of completeness though, we consider the addition of consistency checks in the validation module.

The validation rules to check the integrity of the information contained in the



HRS2 database tables (e.g. location.DBF and individual.DBF) are contained in the Validate specification file found in HRS2.PRJ. The checks display records with invalid data which is unknown at the time of entry or records with invalid data. In some cases the checks will be the same as those in the checkvalidity boxes of the data entry screen.

The validation routines are dependent on the rules for data at your site. The current code distingushes between between location (location.DBF) and individuals (individuals.DBF) checks. Comments have been placed in the code for the routines highlighting where the specific database file checks ought to be placed.

To make changes to the validation routines to report inconsistencies, the steps are:

1. Open the HRS2 project file (HRS2.PRJ) and click on the forms tab

2. Choose and edit the form specification file named VALIDATE.

3. Look at the properties of the form (by right mouse button click on empty region in the form). Click on the methods tab in the properties dialog. The methods at the end are the HRS2 methods used for validation checking. If you click on any of the methods, a brief description of that method is given if the property descriptions are on (if not, right mouse button click in the property window and choose property descriptions).

4. Decide on the appropriate place to put the new validation rules. For example, if the new field was inserted in the location rule, then put in the rule in the locationchecks method. Double click on the method to open up a window to look at the code.

6. Insert the consistency check code after any "if ... endif " statement.

7. Return to the project file by saving all control functions associated with the VALIDATE screen specification

8. Rebuild the application to allow new code to take effect in the HRS2.

The syntax for a validation code snippet related to the Location table is as follows:

If (variable) = Valid Code (or date or range of values) do CheckOutput with m.search_loc, "XX", "Location": Error Message" endif

In the above syntax, variable is a field in the FAMILY database table; the Valid Code could be acceptable values for the variable, a date, or range of values, etc.; CheckOutput is a function for displaying messages on the screen; m.search_fam is a memory variable used in searching for a particular record in the FAMILY database file; XX is a unique number between 01 and 99



associated with each error message reported through the validation routine; and Error Message is a description of the error encountered during the validation check. Error Message can be any text of length 50. HOUSEHOLD is included in Error Message to indicate the database with the inconsistency.

The syntax for a validation code snippet related to the Individual database table is as follows:

If (variable) = Valid Code (or date or range of values) do CheckOutput with m.search_ind, "XX", "Individual": Error Message" fflg = resetflag(fflg, "F") endif

In this syntax, variable is a field in the Individual table; the Valid Code could be acceptable values for the variable, a date, or range of values, etc.; CheckOutput is a procedure for displaying messages on the screen; m.search_mem is a memory variable used in searching for a particular record in the MEMBER database file; XX is a unique number between 01 and 99 associated with each error message reported through the validation routine; and Error Message is a description of the error encountered during the validation check. Error Message can be any text of length 50. Individual is included in Error Message to indicate the table with the inconsistency. The line "fflg = resetflag ..." indicates the result of the validation check by setting a flag attached to the record to fatal (F).

Creating a New Entry Form Page 1 of 2

file://F:\hrsmanuals\NewEntryForm.html 9/15/00

How to create a new data entry form

Data entry forms need to be created when new categories of information need to be collected. Examples include economic social group information, child immunization data, family planning usage, extra information about children, … This section address the issues associated with designing and creating new data entry screens.

The first task is to choose among the three types of base data entry forms contained in the HRS class library. The choice is made based on the type of information that will be entered :

l EntryForm : Data is entered once for each entity and the entity stays relatively constant through time. Examples include the background information about a social group, a list of Primary health care facilities and their attributes, Village Leaders, etc.

l ObjectForm : Data is entered once for each entity and the entity stays relatively constant through time. The difference between Objectform and EntryForm is that this form has fields for the field Worker and the status of the data. If you don't keep track of this information, then use entryform. Examples include the background information about a social group, a list of Primary health care facilities and their attributes, Village Leaders, etc.

l EventForm : Data changes through time and the information to modeled is point in time data. Examples include Vacinnation shots, child health characteristics at a particular point in time, a women's attitudes about family planning, etc.

l EpisodeForm : Data changes through time and the information to be modeled occurs over an interval of time (and the interval is a significant component of the data). Examples include intervals of illness, pregnancy, or contraceptive use.

To specify which form you would like to use requires a few steps :

l Click on View menu option, and then choose Options. l A Dialog box appears with many tabs -- chose the forms tab l In the section title "template classes", click on the form check box and

then click on the command button labeled ... l Choose one of the above form types from the HRS class library (located

in the Classes subdirectory of the HRS2 program).

Now, when you create a new form (by pressing File, New, Form) that form will be the choice you made in the last step of the above procedure.

Creating a New Entry Form Page 2 of 2

file://F:\hrsmanuals\NewEntryForm.html 9/15/00

At this point you will have a form with a control bar and possibly a few textbox fields. Additional textbox fields are needed to collect the rest of the information. But before textbox fields can be added, a data table needs to be created.

The data table may need to have a certain minimum collection of variables based on the type of form chosen :

l Fields needed for Entry Form : none l Fields needed for ObjectForm :

¡ Field Worker ¡ status of data

l Fields needed for EventForm : ¡ Event ID ¡ Field Worker ¡ status of data ¡ type of event ¡ date event occurred ¡ observation ID

l Fields needed for EpisodeForm : ¡ Episode ID ¡ Individual ID ¡ status of data ¡ Event Type that initiated the Episode ¡ Begin date of Episode ¡ Starting observation ID ¡ Event Type that ended the Episode ¡ Ending date of Episode ¡ Ending observation ID

Local Views

Attach to Menu

Bringing Legacy Data into the HRS2 Page 1 of 5

file://F:\hrsmanuals\LegacyData.html 9/15/00

Bringing Legacy data into the HRS2This is one of the more difficult tasks that can be undertaken with the HRS2. A number of problems present themselves and not all of them are technical. First, legacy data has a life of its own, with project staff knowing and understanding the nature of the legacy system. Changing to a new system introduces risk and the lack of familiarity with the new system can, at times, generate resistance from the programming and field staff. Two good reasons exist for changing to the HRS2 :

l Data Integrity : Without strong consistency checking, the weight of bad data tends to sink projects. The HRS2 provides some built in consistency checking and has machinery to support the addition of more checks.

l Reduced work : Some projects essentially do periodic censuses and link the data together afterwards. This involves a fair amount of field work, even more data entry work, and finally an almost impossible task for the computer staff which is responsible for linking the data together. The HRS2 has an initial census, after which only changes are recorded. In most circumstances only a small proportion of the population, approximately one-seventh during a three month interval in one African project, undergoes some demographic change. This means a smaller amount of work in the field collection, data entry, and error resolution.

The most significant reason for not changing to the HRS2 is the technical resource requirements that are needed. Any complex software system requires technical support - the HSR2 is no exception. Software systems have a tendency to break or need modifications and, at the moment, all the up front work done in the HRS2 still does not alleviate the problem of getting reasonably competent programmers to support the system. Sometimes this is done by hiring an on-site data manager, in which case a few years of programming experience and a willingness to learn is essential for the success of the project. Other sites have done well by hiring local consulting groups to support the software.

The following steps outline the process of making a change to the HRS2

l Construct a new data model that merges the old data model with the HRS2 data model. Start by constructing a data model for the current software system (see the HRS2 data model in earlier sections) and then compare this data model to the HRS2 data model. The closer you move to the HRS2 data model, the easier it will be for the transition. Also, carefully considering any decision to drop HRS2 fields or tables. This diagram was carefully constructed and reflects years of experience from three systems developers working on these problems. If you decide to change variable names, this will introduces more of a nuisance - but good reasons exist for keeping old variable names

l Modify the WHRS database in the HRS2 project to fit the new data model.



l Consistency checks need to be performed on the legacy data. The best solution at this stage is to completely clean up the legacy data so that no inconsistencies exist. In this case, the validation rules can be attached to the database table and the amount of data entry form programming is reduced. In many cases, it is impossible to resolve inconsistencies by the time the software is in place. Some consistency checks will need to be placed in the data entry forms and the project will be in a position of having new data conform to the consistency logic while the old data still has problems.

l Build data entry forms and adjust menu entries. Of course, if the merged data model is close to the HRS2 data model this task is easier.

This chapter focuses on the construction of the new database and issues of performing consistency checks on legacy data. Step 1 requires some real careful decision making and I can think of no algorithm except to make it as close to the HRS2 as you comfortable with and feel free to email me (Bruce MacLeod : [email protected] ). Step 4 is covered in various sections of this document.

Modifying the WHRS database

The HRS2 database contains not only a list of database tables, but field rules, field descriptions, table level rules, SQL Views, and stored procedures. Therefore, it is best to alter and then put your data into the HRS2 database. It would be fair more complex to construct a new database from scratch.

Start by changing all relevant field names. If you change any of the key ID field names, be forewarned that this will involve a substantial amount of changes to the underlying system. You can change the data type of the ID fields or the length and this will involve a small amount of changes, but changing the name of an ID field will require changes to the numerous locations in the code that this ID is referenced. Examples ID field names include : locationid, individid, observeid, eventid, …

Change any of the consistency rules that are not appropriate- both at the field level and in the stored procedures. You should strive for putting a rule in each field of the database.

If you load your data by append from command, then it may be necessary to disable the triggers on the four event tables : inmigration, birth, outmigration, and death. The triggers on these tables construct or delete new records in the episode tables for the event. This keeps the event and episode data consistent. If you are constructing the episode tables by hand, then you will want to blank out the triggers (under the table option when modifying the table) while loading the data.

You may also want to blank out the row level checks for the table (under the



table option when modifying the table). Row level checks involve consistency rules for multiple field within a table or consistency between fields in the new record and other tables. Sometimes legacy data will not pass the consistency rules and you want to load it any way. The next subsection describes how to identify those records that do not pass the checks.

Once all the data is load, you may want to browse all the records to make sure all fields are filled in the appropriate ways. The next section describes how to improve the quality of the data.

Consistency Checks on Legacy data

The motivation for strong consistency checking is clear for the project manager : good data, good analysis, and subsequent success in research and publications. There is a very clear rational for programmers and data managers to want strong consistency checking as well : when bad data is transferred from the field worker to the computer, then it becomes the data managers or programmers problem - not the field workers. Enormous amounts of time can be eaten up by resolving bad data problems that should never have been put in the computer in the first place. So this task, while it may seem cumbersome at first, can make the environment for programmers and data managers far more rewarding. Work will involve more programming and analysis instead of tracking down data inconsistencies. Correcting bad data requires a commitment from project directors, field workers, and the data management staff.

There are a few techniques to help track down inconsistencies. The first goal is to develop field level rules for every field in the database. Some fields that allow one value from a set of values are relatively easy to write. An example is gender in the individual table. A field rule such as

gender="M" or gender="F"

can be inserted into the field rule box (which you get access to when you modify the individual table and click on the gender field). You will want to put an error message in the Validation text box for the situation when the rule fails. Suppose that this rule fails - somehow the legacy data contains values other M and F for the gender field. You must erase the rule to get out of the table modification dialog box.

If a validation rule fails, then it is necessary to track down all those records that failed the rule. This can be done in a number of ways. One way is to use the browse command :

select individual browse all for not (gender = "M" or gender="F")

Another way is make a copy of those records that flunk the consistency check. The records could then be used a report that could go out to the field workers. To make a copy of those records that flunk the gender consistency check :



select individual copy all to c:\temp\badgender for not (gender = "M" or gender="F")

This data can be printed out in a raw form or sent through a report.

Many of the functions in the stored procedures of the database will help in writing field level rules. Examples of procedures to identify the existence of IDs include : individidfound(), locationidfound(), oobserveidfound(). Other related procedures are legalfemaleid(), legalmaleid(), legalmotherid() or legalfatherid(). Before going any further, verify that these procedures work in the way they are supposed to. An validation rule that restricts outmigrant records to have a legal individual ID can be placed in the rule box associated with individid in the outmigrant table :

individidfound(individid)

This rule takes a long time compute as it goes through the individual file for every record in the migration file. For one record it is reasonably fast though. If this rule fails, then the above browse and copy commands can be used.

After putting all the field level rules in, the next step is to try to put the record level rules in. For each table, there is a stored procedure which has the table name followed by the text "rowchecks". So, for example, record level checking for the individual table is in a procedure called individualrowchecks(). To put this procedure in place for the individual table, you modify the table, then choose the table tab, and then put the individualrowchecks() procedure name in the record validation rule box.

Suppose that the row validation rule fails for some records, then you can use the above browse and copy commands to isolate the problem records.When using the browse and copy commands, it is difficulty to determine which one of the row level rules caused the row check to fail. This display command can be used to display the error message:

select individual display fields individid, whrserrordisplay() for not individualrowchecks()

The output can be set to a printer with the set printer on command. Alternatively, you can send the output to a file with the set alternate command.

Consistency Checks put in Forms

You may not be able to put all consistency checks into the database table rules as it takes time to resolve data problems. Consistency rules can also be placed in the data entry forms. This will restrict new data to legal values. Generally, these rules are placed in the checkvalidity method of the control



that will enter the data. The process of writing these consistency checks are described elsewhere in this document.

Page 1 of 1

file://C:\WINNT\Profiles\csavel.000\Desktop\ss.html 9/15/00

The HRS Technical Manual

How to Link Special Studies into the HRS2®

The HRS is responsible for keeping track of core demographic information. This includes births, deaths, and migrations. In most cases a project will want to maintain information in addition to the demographic core. Sometimes the information can be included as part of the HRS without much difficulty. For example, if a project wants to maintain religious affiliations, then a single field could be added to the individual information. In most cases, additional data need to be maintained separately from the HRS2 database. For example, separate data systems are needed for maintaining information about hospital visits, fevers in children, or contraceptive use. We do not recommend changing the HRS2 to accommodate a special study. A special study data system can read HRS2 data for consistency checking and searching without adding data to the HRS2. This approach will allow special studies to rely on the HRS2 core without cluttering up HRS2 data. An example of a special study is included in the HRS2.1 distribution.

Documents

THe HRS2 Technical Manual - Population CouncilThe Project organizes and provides access to€ HRS2 files Object Oriented Design in HRS2 ... Modifying the WHRS database Consistency