by David G. Theriault, Nicola Terry, Cees van der Leeden and Richard G. Newell

For the past 15 years GIS users who enthusiastically embarked on grandiose projects
have, with very few exceptions, spent enormous amounts of time and effort mired
in the initial phase of data capture. The organisations worst affected are those
where the capture effort is concerned with the conversion of old documents to a
digital, structured database, e.g. utilities, local authorities and cadastral agencies.
The reasons for the difficulties are mainly inappropriate tools being applied to
the problem. This paper presents an approach in which the problem has been completely
re-examined at four levels. The first consideration is at the project management
level where a proper analysis of goals and targets is required. The next level
is concerned with the rapid development of specific applications in an ergonomic,
integrated object-oriented environment. These applications must be supported by
facilities for rapid take-on of data (e.g. raster-vector integration, interactive
vectorisation tools and multiple grey-level images) and a topologically structured,
seamless spatial database. Finally, powerful workstations with a visually pleasing
window environment will be needed. Benchmarks so far seem to indicate that factors
of 3 to 5 improvement in time may be achieved.

Introduction

Everybody now recognises that one of the largest costs (possibly the largest cost)
of implementing a GIS is data conversion. In particular, the problem of converting
data held on maps into a database containing, points, lines and polygons which
are intelligently structured, is particularly burdensome. Much effort has been
invested in trying to improve the process beyond the traditional methods based
on manual digitizing. These include the use of scanned raster data, automatic conversion
of raster into structured vector, automatic symbol and text recognition and “heads
up” (on-screen) digitizing. All of these have their place, but we will advocate
in this paper that other aspects of a system’s function-ality and its structure
can have more dramatic effects in reducing conversion costs in many cases. Our
experiences stem from applying these methods to the capture of utility foreground
data and cad- astral polygon data, where automatic approaches based on artificial
intelligence break down.

Putting it another way, it seems that, with the current state of the art of automatic
recognition methods for data conversion, there will always be a large proportion
of documents which can only be interpreted and input by human operators. The exercise
then becomes one of providing the best possible environment for humans to operate.
This includes, not just providing much improved user interfaces, but also the very
structure of the system itself. Badly structured, poorly integrated systems contribute
greatly to the costs of conversion.

We contend that data capture is not a separate function carried out in isolation
of the GIS it is to feed, but the data capture system itself should embody much
of what one would expect from a modern corporate GIS.

Goals and Targets

It is important at the start of a GIS implementation to identify what it is that
the GIS is intended for. This may be an obvious thing to say, but it is amazing
how in many cases in the past, the conversion exercise has borne no relationship
to any pre-assigned goals. An organisation embarking on GIS can define the business
case for the whole project. This will result in:

• identifying the end users of the system (including those who might
• never actually sit down in front of a terminal).
• a detailed breakdown of activities on the system.
• an outline of possible future requirements.

From the analysis it is possible to know the exact data requirements, including
the coverage required and the level of intelligence required in the data structures.

This business case analysis has not always been done rigorously. The result of
this is that frequently far more is converted than is strictly necessary. It does
not seem very many years since the silly arguments about the relative merits of
raster and vector. Now that the discs are cheap and the processors and displays
are fast, everybody recognizes that a raster representation for a landbase is an
excellent and cost effective way to get started. Raster not only provides a perfectly
adequate representation for producing displays and plots, but also is a good basis
for the capture of intelligent data.

For a utility company, the data may be divided into background and foreground
data. The source of the background data is good quality paper maps from the national
mapping agency (e.g. the Ordnance Survey in the UK and the IGN in France). A complete
digital vector representation of these background maps is usually not available
or is expensive, so we would advocate in this proposal that these maps should be
scanned using a good quality scanner. The cost of producing a scanned background
mapbase is a small fraction (perhaps one tenth) of the cost of producing a vector
background mapbase.

It is thus essential to decide at an early stage, what are the objects in the
real world that need to be represented in the system, in order that the foreground
capture can be limited to these.

After all, a GIS is no more than a corporate database management system, in which
one of the query and report environments is a map. Thus the same data modelling
and analysis that goes into DBMS implementation needs to be done for GIS implementation.
This data model will determine that which needs to have an intelligently structured
graphical representation and that which does not.

It is essential that goals are set so that a complete coverage of a defined and
useful area can be achieved in about one year to 18 months. If the time frame is
longer than this, the management will get impatient to see results, the maintenance
problem will overtake the primary data-capture and the whole thing will fall into
disrepute. We would go as far as to say, if the organisation cannot commit the
money and resources to achieve this, then it should wait until it can.

Development of Specific Applications

For the purpose of the data conversion project, specific applications for input
and edit need to be developed for each data item. In parallel, it is necessary
to implement and test the end applicat-ions of the system. Traditionally in a GIS
project, these tasks have consumed an enormous amount of time and valuable resource.
For sound reasons, the data capture project is postponed until these applications
show robust results. The advances that have been made in data processing and database
management, such as rapid prototyping and 4GL environments, have only recently
surfaced in some commercial GIS systems.

This task of applications development is made much easier if the GIS already has
an extensive library of generic functions which can be applied to the specific
data model implemented. These functions might include generic editors for geometry,
attributes and topology, network functions as well as polygon operations including
buffering and overlay. It is in this area that interactive object-oriented environments
really come to the fore as new functionality can inherit much of the existing functionality
of a generic system as well as providing very rapid prototyping and development
capabilities.

Integrated System Approach

This section contains the main burden of our argument. In the past, most data
capture was carried out using conventional digitizers on single user systems, one
sheet at a time, in isolation of the GIS. We contend that data capture should be
carried out using on-screen digitizing in a multi-user environment using a continuous
mapbase in the environment of a fully integrated GIS.

Such an approach gives considerable savings in time and cost in making a complete
coverage. The basis of the approach is to eliminate many of the activities which
take time in systems which are not well integrated.

The procedure starts, by capturing an accurate background landbase. This may well
be a base constructed of vectors and polygons, if it is available, such as the
data available from the Ordnance Survey. Alternatively, a base constructed of scanned
raster is much cheaper. Scanning at 200 dots per inch is barely adequate for this
purpose and it is worth the marginal additional expense of scanning at a higher
resolution of say 300 or even 400 dots per inch. With modern compression techniques,
400 dpi only takes twice, not four times, the space of 200 dpi. Semi- automatic
vectorisation (see below) is considerably improved at the higher resolutions.

[ Figure 1 not available ]

Figure 1: Basic Data Model

The next stage is to input data which can be used to locate objects and mapsheets
as well providing an essential geometric reference base. This includes such things
as street gazetteers, map sheet locators and street centre-lines. Thus locating
any street or map sheet and getting it onto the screen should be very efficient.
Conversely, the system should be able to identify the mapsheet(s) containing any
point or object indicated on the screen. Some locational data may already exist
in existing company databases. Doing this before the foreground capture can improve
overall efficiency.

Finally, the source maps containing the foreground data of interest can be scanned
and the process of on-screen data capture can begin in an environment in which
many of the hurdles in conventional systems are removed. Figure 1 illustrates the
order in which information is assembled.

The following is a discussion of several important issues, including:

• On-screen digitizing
• Digitizing across many sheets without interruption
• Geometry, topology and attribute capture in one integrated GIS environment
• Multi-user data capture into one continuous database
• The full facilities of a GIS for validation and running test applications

On-screen Digitizing

The conventional way of digitizing map data is to use a tablet or digitizing table.
While, for certain kinds of data, this is usually of sufficient accuracy, a significant
amount of time is taken by the operator in having to validate the input on a work-station
screen, which is a separate surface to that on which the digitizing is performed.
Thus it is difficult to check for digitizing errors and also for completeness,
as the two images are not together.

Further, it is common for there to be minor mis-alignments between a quality background
map sheet and the map sheet on which the foreground data is drawn. With on-screen
digitizing it is possible to do a quick local realignment to compensate for such
errors.

The process can be considerably improved with facilities for semi-automatic vectorisation
of the raster data, so that high accuracy can be obtained while working at a relatively
small scale. This reduces the need to zoom in and out frequently as the automatic
vectorising facilities are usually more accurate than positioning coordinates by
eye.

It is also necessary to be able to handle greyscale scanned maps in cases where
the source maps are of low quality, or contain a wide variety of line densities,
such as pencil marks on a map drawn in ink. Compression techniques should allow
a large number of greyscale or colour maps to be stored efficiently.

Digitizing Across Many Map Sheets Without Interruption

With conventional digitizing, it is not possible to digitize at one time any object
which lies across a map sheet boundary. Objects such as pipes or electricity cables
frequently cross map sheet boundaries at least once. In these cases, the parts
of the object must be connected together by an additional interactive process.
Ad hoc techniques such as “off sheet connectors” and artificial segmentation of
the object must be avoided.

With on screen digitizing, the process starts by scanning all (or at least a significant
number of) adjacent foreground maps into the database before digitizing commences.
This, in effect simulates a single large continuous raster mapbase. Thus the digitizing
of complete objects can be achieved without interruption, as all of the information
is presented together on the screen about the object.

Geometry, Topology and Attribute Capture in one Integrated GIS Environment

It is also necessary for the input, edit and query functions of the GIS to be
available to the operator during data capture. Thus, little time is wasted moving
from one user interface environment to another. Objects with similar attributes
to objects already in the system can use those attributes as a starting template
for modification.

The integration of the digitizing capability with the geometric CAD type facilities
means that constructive geometry can be mixed with digitized geometry at will.
Further, the integration of the geometry capability with the topology facilities
means that most topology can be generated automatically by the system at the time
that objects are positioned on the screen. For example, a water valve placed on
a water pipe causes a node to be inserted in the water pipe, and the correct connectivity
to be inserted in the database.

The inclusion of gazetteering functions considerably reduces the amount of time
spent locating objects in the database.

Multi-user Data Capture into one Continuous Database

Approaches based on digitizing one sheet at a time by means of single user data
capture systems result in a lot of time being spent merging and managing the many
independent data files generated. The problem becomes particularly acute when edits
are required to data which has already been input.

The system should permit many users to simultaneously update one continuous mapbase.
This can only be achieved efficiently if the system allows any number of versions
and simultaneous alternatives to be held in the database. This results in making
the task of administering many users engaged in simultaneous additions and modifications
to the database much easier.

This single issue is one of the dominant reasons why data capture can be very
time consuming and costly.

The GIGO myth

A popular cliché these days is the GIGO (garbage in garbage out) story. Well
it is not actually true in many cases, or at least the situation may be recoverable.
It is our experience that many of today’s GIS do in fact contain so- called garbage.
The reason we say this is because frequently the geometry is good enough, but the
structure of the data is defective, often objects are split across map sheet boundaries
and object attributes are wrong or just plain missing. In other words the data
cannot be used for the purpose it was intended.

In our experience, there is usually enough in these databases to convert the data
into a satisfactory quality by a process of “laundering”. This is a process where
the data is imported into a GIS environment, and automatic processors can be applied
to the data to infer missing topology and missing attributes and a range of validators
can be run to detect and highlight problems graphically for a human operator to
deal with. The cost of doing this is usually very much less than the cost of the
original data capture, but it does require a rich GIS environment in which to do
it.

It is particularly useful if the system contains rapid prototyping and development
facilities so that it makes the writing of special programs to clean up particular
datasets more feasible. These programs may well be used only once, then thrown
away.

User Interface and Platform Issues

Much of what has been proposed above has only been made possible by the huge improvements
in hardware and software in the last three years. On the hardware front, the drop
in price combined with the huge increases in performance, memory and disk sizes
means that data capture can be carried out in the environment of a large seamless
database, managed by a modern GIS. The improvement in emerging user interface standards,
coupled with the arrival of object-oriented development environments for the rapid
development of custom built user interfaces for particular data capture tasks is
another contributing factor.

Conclusion

Whereas most effort in the past in improving data capture has been expended on
fully automated methods for converting source map data into a structured data base,
we believe that for many types of documents, the human operator cannot be eliminated,
but the present situation can be considerably improved by providing a fully integrated
system.

Many of the hurdles which contribute to the cost of data-capture can be eliminated
by several levels of integration including:

• The integration of source and target images in one database and on one screen
  via the use of raster scanned maps.
• The integration of all map sheets in one seamless mapbase.
• The integration of map sheets and locational data.
• The integration of data capture, validation and GIS functions in one environment.
• The integration of many users capturing data into one version managed multi-user
  database.

As well as these issues of integration, the application of robust intelligent
techniques, such as semi-automatic vectorisation and automatic topology generation
further improve the process. Finally, the advent of modern user interfaces and
object-oriented development environments lead to the production of optimised custom
built user interfaces ideally tuned to the task in hand.