Equipped for the Future Research Report: Building the Framework, 1993-1997 Document Body Page Navigation Panel

Equipped for the Future Research Report: Building the Framework, 1993-1997


Page 1 2
Equipped for the Future Research Report
National Institute for Literacy NIFL
By Juliet Merrifield
EFF Technical Report
Building the Framework, 1993 Đ 1997 1.
1 Page 2 3
Equipped for the Future Research Report
National Institute for Literacy NIFL
By Juliet Merrifield
March 2000

EFF Technical Report
Building the Framework, 1993 Đ 1997 2.
2 Page 3 4
Equipped for the Future publications are available from ED Pubs:
Publication No.
EX 0019P
Equipped for the Future: Customer-Driven Vision for Adult Literacy
and Lifelong Learning, July 1995

EX 0020P Equipped for the Future: Reform Agenda for Adult Literacy and
Lifelong Learning, February 1997

EX 0094P EFF Voice: Equipped for the Future, Vol. 1, No. 1, Fall/ Winter 1999
(Newsletter)

EX 0099P Equipped for the Future Content Standards: What Adults Need to
Know and Be Able to Do in the 21st Century, January 2000

EX 0106P Equipped for the Future Research Report: Building the Framework,
1993 Đ 1997, March 2000

EX 0110P Equipped for the Future Assessment Report: How Instructors Can
Support Adult Learners Through Performance-Based Assessment,
July 2000

To Order:
1. Call toll free: 1-877-4ED-Pubs (1-877-433-7827)*
2. Use the Internet: http:// www. ed. gov/ pubs/ edpubs. html
3. Send e-mail to: edpubs@ inet. ed. gov
4. Ask for Publication No.: (see above)
*If 877 is not yet available in your area, call 1-800-USA-LEARN (1-800-872-5327);
TTY/ TDD call 1-800-437-0833. 3.
3 Page 4 5
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1
SECTION 1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
The EFF Process 1993-1997 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
The Impetus for EFF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
Approaches to System Reform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

SECTION 2
EFF's Approaches to Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

SECTION 3
Concepts and Theories in EFF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Purposeful View of Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Rooting Education in the Context of Lives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Application, Not Possession, of Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Adult Development as Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12

SECTION 4
The Foundation: Purposes for Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Process for Learner Consultation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Analysis of Learner Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Four Purposes for Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15
Feedback and Validation of the Purposes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
The Importance of the Purposes for Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17

Table of Contents
Equipped for the Future Research Report 4.
4 Page 5 6
SECTION 5
Role Maps for Effective Citizens, Workers, Parents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Raw Materials for the Role Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Developing Draft Role Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Structured Feedback to Revise and Validate the Role Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
The Inquiry Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Revising the Role Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
Role Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27

SECTION 6
Identifying Skills and Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Developing a Database on Skills and Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Coding the Skills Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Linking Skills with Role Map Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32
Common Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Generative Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
Knowledge Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36

SECTION 7
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38
Integrating Theory and Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38
Learning From the Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38

Footnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42
Appendices
Appendix A: Planning Projects, 1995-96 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44
Appendix B: Structured Feedback Process, 1996-97 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
Appendix C: Documentary Sources for Skills and Knowledge Database, 1997 . . . . . . . . . . . . . . . . . . . . . . .52
Appendix D: EFF Generative Skills Descriptions (revised, May 1999) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54

Equipped for the Future Research Report 5.
5 Page 6 7
1
Equipped for the Future Research Report
DECLARE AN INTEREST IN EQUIPPED FOR THE
FUTURE (EFF). From the time in 1993 when I
first heard about the learner consultation, this
project has seemed exciting and innovative
to me. The Center for Literacy Studies (CLS), where
I worked at the time, helped distribute the discus-sion
guidelines to programs in Tennessee, and
encouraged them to get their students writing. Later
I was to play a small role during the planning year,
when CLS, led by my colleague Brenda Bell, was one
of eight EFF planning groups. After I left CLS in the
summer of 1996, I was asked to synthesize data for
the citizen role from across the planning projects.
CLS has continued to play an important role in
EFF's development, which I have only watched from
across the ocean in England.
My role in writing this research report was not
to pretend to be a completely independent observer.
My early involvement in the EFF research gave me a
particular perspective, not just a reporter's but a syn-thesizer's.
I am pleased to have been invited to devel-op
this first research report, not least because it gave
me an excuse to find out what has happened since I
left the U. S. I have not been disappointed. EFF has
held to its early promise to offer a new way of think-ing
about adult education. It has, perhaps uniquely,
managed to work in a participatory way with the
field at the same time as promoting leading-edge
development. It has given practical focus to newer
theories about learning and adult development. The

expansion and growth of interest, support, and
involvement from across the country testifies to the
power of an idea and the value of participation.
In preparing this report I have had access to all
the project's internal documents. I also interviewed,
in person or by phone, the staff leading each of the
role development efforts and many of their col-leagues
who were involved in structured feedback
and coding data. Four years' work yielded a lot of
paper. Inevitably during that time the conceptions
of the task were refined and definitions clarified. I
have tried to show these changes over time without
making the report too difficult to follow.
The report could not have been done without
the guidance of Sondra Stein as the National Insti-tute
for Literacy's project director of EFF; Brenda
Bell, Associate Director at the Center for Literacy
Studies at The University of Tennessee/ Knoxville
and coordinator of the citizen role work; Lisa Levin-son,
project director at the Center for Adult Literacy
and Learning at the University of Maine and coordi-nator
of the worker role; and Meta Potts of the
National Center for Family Literacy and coordinator
of the parent and family member role. Professor Hal
Beder of Rutgers University Graduate School of
Education and Dr. John Comings of the National
Center for the Study of Adult Literacy and Learning,
Harvard University kindly commented on an earlier
draft. Errors that remain are mine.
-Juliet Merrifield, October 1999

Preface
I 6.
6 Page 7 8
HE EQUIPPED FOR THE FUTURE (EFF) PROJECT
of the National Institute for Literacy (NIFL) is
working toward system reform for adult liter-acy
and lifelong learning. At its heart lies a
clear vision of what adults need to know and do in
order to fulfill their roles as citizens, workers, and
family members.
Through EFF, NIFL has sought to build consen-sus
on the 'big picture' guiding policy and practice,
and to develop content and performance standards
that link the classroom with individual and societal
expectations. The goals are ambitious: to shape an
education system whose goal is not to remedy defi-ciencies
from earlier educational experiences, but
"to prepare adults for the future - to build on
what they have already learned through experi-ence
as well as formal education, to prepare
them for new, unanticipated responsibilities in
the present, and to provide them with the tools
to enable them to continue to learn."
(Stein,
1997: 1).
This report focuses on the research aspects of
EFF - the process of gathering and analyzing data to
create, refine, and validate the framework from 1993
through the summer of 1997. EFF is not simply a
research project, although the framework is based
on original research: from the beginning it has
sought both to create a new framework for adult lit-eracy
education and to develop organizational sup-port
for it.
The purpose of this report is to document the
research conducted through the summer of 1997
and the concepts and theories involved. It focuses
less on the products (which are detailed in a series of
other reports Đ Stein, 1995 and 1997; NIFL, 1998),
than on the process - on what was done and why.

But in examining the process, certain themes
become evident:
° EFF Integrates Theory and Practice
-EFF builds on and contributes to the growing
body of approaches to learning as a purposeful
act - not decontextualized and value-free, but
embedded in particular purposes and specific
contexts.
-EFF's focus on the application rather than the
possession of skills and knowledge is a contri-bution
to adult education - following in a long
tradition that needs to be revived.
-The EFF project represents the first time there
has been a concerted attempt to map the major
adult roles. Although a great deal of work had
been done around the worker role, through
SCANS, O* NET, and the occupational skills
standards, much less had been done on the citi-zen
and parent/ family member roles.
° EFF Takes an Iterative Approach
to Theory Building
-The process of EFF has been one of the most
extensive consultation and participation process-es
ever carried out in adult education.
-EFF staff have tried to keep a holistic view of
where the project is going while at the same time
working intensively on small pieces of the frame-work.
Responsiveness to constituents who have
some stake in the outcome means the whole is
always being modified by the development of the
parts. EFF adopted this iterative approach in
order to generate system reform that would be
credible and acceptable to the field of practice, to
stakeholders, and to policymakers.
EFF's most significant accomplishment has
been to shift thinking about the purpose of adult

3

Equipped for the Future Research Report
Introduction
T 7.
7 Page 8 9
education. From a conception that adult education's
work is to replicate K-12 education and teach
knowledge and skills that are autonomous and inde-pendent,
EFF has pushed us toward thinking of
adult education as preparing people for the future
by teaching integrated skills and knowledge needed
to be more effective workers, parents and citizens.
Equipped for the Future has been a unique and
remarkable effort to model adult education for the
next century.

The EFF Process 1993-1997
The EFF initiative began with an invitation to adult
learners to help shape adult education policy by
writing about what Goal 6 of the National Educa-tion
Goals meant to them. 1 This primary goal for
adult literacy and lifelong learning states:
"By the year 2000, every adult American will
be literate and will possess the knowledge and
skills necessary to compete in a global economy
and exercise the rights and responsibilities of
citizenship."
Learner consultation was the first step in an
effort to make Goal 6 a real guide for the literacy
field, one that clearly describes what it means to be
literate, to compete in the global economy and
exercise the rights and responsibilities of citizen-ship.
This consultation shaped the first stage of
Equipped for the Future (1994-1996), the develop-ment
of a framework for standards-based system
reform. In subsequent work to develop the frame-work,
EFF consulted with a wide range of stake-holders
to validate the four overarching purposes
for learning identified from the learner consulta-tion.
During this planning year, projects also devel-oped
guiding principles for the work and for the
standards to be developed.
The second stage of EFF (1996-1998) was
developing the content standards. Planning projects

conducted research to identify what adults need to
know and be able to do in order to perform effec-tively
their key life roles as citizens, workers, and
parents, and used this research to develop and vali-date
"role maps." These were linked with skills and
knowledge identified from literature reviews. Com-mon
elements across the roles, including activities,
skills, and knowledge, formed the basis for content
standards, which were then field tested by 25 local
field development partners.
In the third, and current, stage of work (1998-
2001), EFF is developing the performance continu-um
for these standards. The process of testing and
refining content standards and performance levels
will be the subject of future research reports.

The Impetus for EFF
A series of critical studies in the early 1990s high-lighted
the need for fundamental reform of adult
literacy education. Successive reports revealed the
significant need for literacy education services, the
inadequacy of the present basic skills system, and
the lack of agreement on vision. NIFL responded to
all three issues in EFF.
The National Adult Literacy Survey (NALS)
showed almost half of adults in the U. S. possessed
literacy skills below the threshold of successful adult
performance (level 3 on the NALS five-level scale).
Performance at the lowest levels was highly correlat-ed
with poverty and underemployment (Kirsch et
al., 1993). The International Adult Literacy Survey
compared literacy in eight industrialized countries
and demonstrated that the US had the highest per-centage
of workers scoring at the lowest literacy lev-els
(Fellegi and Alexander, 1995).
The National Evaluation of Adult Education
Programs pointed out inadequacies in the system
intended to address adults' skill needs: most learners
stay in programs a very short time; those who stay

4

Equipped for the Future Research Report
EFF's most significant accomplishment
has been to shift thinking about the purpose
of adult education.
8.
8 Page 9 10
may make initial small learning gains, but not a lot
of long-term literacy skills gains (Young et al.,
1995). In the same year, the General Accounting
Office reported to Congress that performance data
from adult programs are unreliable and incomplete
(Weiss, 1995). It noted both practitioners and
researchers distrusted the most commonly used
tools for measuring performance and progress.
The GAO was particularly concerned about
the difficulties in evaluating the effectiveness of a
system whose objectives are not clearly defined. The
1966 Adult Education Act, which regulated the field
for over 30 years, had a broad and loose vision, as
the GAO summarized it:
"to improve educational opportunities for adults
who lack literacy skills necessary for effective citi-zenship
and productive employment Éto
encourage the establishment of adult education
programs for adults to (1) acquire basic skills
needed for literate functioning, (2) acquire basic
education needed to benefit from job training
and obtain and keep productive employment,
and (3) continue their education to at least the
secondary school level."
(Weiss, 1995: 14)
There was no agreement on the meaning of "literate
functioning," or on the literacy skills "necessary for
effective citizenship and productive employment."
NIFL was charged by Congress with measur-ing
and tracking the progress of the nation in
meeting Goal 6 of the National Education Goals.
While the NALS provided a profile of the literacy
skills of the population, there was no consensus on
what literacy skill adults need. Goal 6 presented not
just a technical challenge in terms of measurement
but a conceptual problem - what does it mean to
be literate? If NIFL's mission was to enhance the
effectiveness of the adult education system in
meeting its goals, then NIFL had to work with the
field to identify and agree on those goals.

Approaches to System Reform
EFF builds on a substantial body of earlier work on
quality and continuous improvement in the private
sector, reinventing government and performance
accountability efforts in the public sector, and stan-dards-
based educational reform. All of these have in
common a focus on results as the driver of system
reform.
Private sector: Total Quality Management
(TQM) and related approaches to quality and con-tinuous
improvement have spread widely in the
business world. In contrast to traditional approach-es
to quality control, which monitor results at the
end of the production process, in TQM results are
monitored at each stage of production. Improving
production processes depends on a clear under-standing
of desired results, detailed analysis of each
step in the production process, and continuous
feedback on how each impacts results (see, for
example, Deming, 1986; Senge, 1991; Stagg, 1992;
Stein, 1993). Continuous improvement efforts
involve workers in monitoring inputs and outputs,
assessing quality, and evaluating production.
Performance accountability in government:
Government reform initiatives began at the state
level in the 1980s. Oregon mounted an initiative
that engaged citizens and organizations throughout
the state in consultation about goals and bench-marks
for government (NIFL, 1995a). Performance
accountability later spread to the federal level,
spurred on by Osborne and Gaebler's book, Rein-venting
Government
(1993). Government reform,
like TQM in the business world, emphasizes "cus-tomer"
needs, clarifying and agreeing on desired
"results," and measuring "return on investment."
(Behn, 1993; National Governors' Association, nd;
Brizius and Campbell, 1991)
Standards-based educational reform: Recent
educational reform, initiated by A Nation at Risk,

5

Equipped for the Future Research Report
To enhance the effectiveness of the
adult education system NIFL had to work with the field
to identify and agree on those goals.
9.
9 Page 10 11
the 1983 report of the National Commission on
Excellence in Schools, has focused on the content of
education - what should be taught and learned, and
what should be assessed. As states moved to upgrade
their core academic requirements, President Bush
and state governors launched a national movement
for developing content standards in key subjects,
and consortia of scholars and teachers began con-sidering
what was most worth learning (Gagnon,
1995). The first of these efforts was independently
initiated by the National Council of Teachers of
Mathematics. (National Council of Teachers of
Mathematics, 1991; Romberg, 1993)
Educational standards identify the end results
of teaching and learning. Efforts to develop stan-dards
for kindergarten through grade 12 (K-12)
are quite diverse but share some characteristics.
They are subject-based - the aim is to clarify and
agree on a body of knowledge that should be mas-tered -
and their main architects have been teach-ers
and academics within that subject area. They
identify what an educated high school graduate
should know.
Occupational skill standards: These started
outside of education, driven by business needs of
employees for particular skills and attributes. The
SCANS report, What Work Requires of Schools (Sec-retary's
Commission on Achieving Necessary Skills,
1991) was followed by development of occupational
skills standards that identify skills and knowledge
needed for particular job clusters. Again, the
method was to focus on desired results as a means
of increasing effectiveness.
The 22 occupational skills standards projects
chose divergent paths, and used different terms and
formats, but with some common approaches. They
all started with the "customers" (however defined)

and their needs, so that the purpose for the stan-dards
was clear. All involved stakeholders in devel-opment
and feedback. All focused on performance
of tasks rather than mastery of content knowledge.
Standards, whether educational or occupa-tional,
are intended to bring about system change,
but they do not do so alone. Clarifying and agreeing
on the desired results of education is the first step.
Comprehensive system reform requires aligning the
whole system - teaching, staff training, assessment,
and reporting - to achieve better results.
In planning EFF, NIFL also drew on its own
experience in systemic change in adult education
through its Performance Measurement, Reporting,
and Improvement System (PMRIS) initiative in the
early 1990s (NIFL, 1995a and 1995b). Five state-level
performance accountability projects supported
by NIFL worked to develop interagency agreement
on shared outcomes for the adult education and
employment training system and on common
reporting systems that would enable these results to
be tracked. The experiences of the PMRIS states
highlighted for NIFL the importance of combining
accountability work at the policy level with bottom-up
work at the class and program level (Swadley and
Ziolkowski, 1996).
NIFL recognized that system change had to
involve the whole system. EFF needed to work at the
national level, with partners and allies having com-mon
or connected policy interests. But it also had to
work at the local level, with program administra-tors,
teachers, and students, to ensure that reform
was practical and applicable. EFF needed to create a
"big picture" that linked teaching and learning with
broad social purposes for education. Creating that
picture had to start with "customers," stakeholders
and practitioners.

6

Equipped for the Future Research Report
Comprehensive system reform requires aligning the whole system -
teaching, staff training, assessment, and reporting - to achieve better results.
10.
10 Page 11 12
QUIPPED FOR THE FUTURE'S APPROACHES
to research have been shaped by the
purpose that inspired that research.
Because the task was to create a
framework that could support system reform, the
research processes used to build that framework had
to be both transparent and inclusive:
° The need to look outside the field for guidance on
defining the ultimate, social purposes of educa-tion,
meant that a broad range of people were con-sulted
in the research phases.
° The need to work simultaneously at the policy
level (on accountability systems) and at the pro-gram
level (on teaching and learning) meant that
the research had both to gather new data and to
link it with existing policy and program tools.
° The need to make the new framework practical
and comfortable for teachers meant that local pro-grams
had to be involved in creating it.
° The need to generate and profit from broad sup-port
meant that each stage had to be iterative:
gathering information, processing it, presenting it
for feedback, and revising it.
EFF's iterative approach has much in common
with "grounded theory" -" the discovery of theory
from data, systematically obtained and analyzed."
(Glaser and Strauss, 1967: 1) This approach to gen-erating
theory results in concepts (or categories)
and hypotheses (or relationships between cate-

gories) that are drawn from the data but have a life
beyond it. The acid test is the usefulness of the theo-ry
proposed. In their formulation of "grounded the-ory"
Glaser and Strauss say:
In discovering theory, one generates conceptual
categories or their properties from evidence; then
the evidence from which the category emerged is
used to illustrate the concept. The evidence may
not necessarily be accurate beyond a doubt (nor
is it even in studies concerned only with accura-cy),
but the concept is undoubtedly a relevant
theoretical abstraction about what is going on in
the area studied. Furthermore the concept itself
will not change, while even the most accurate
facts change. Concepts only have their meanings
respecified at times because other theoretical and
research purposes have evolved. (ibid.: 23)
Grounded theory can use different research
methods, but is particularly suited to qualitative
methods, such as those used in EFF, because of
their rich depiction of human beliefs and actions
in social and cultural contexts. Within the broad
field of qualitative research methods, EFF drew
primarily on naturalistic approaches. Guba and
Lincoln describe the naturalistic approach to eval-uation:
it "moves through several iterations; it
makes credibility checks possible at each stage and
invites negotiation on points of difference." (Guba
& Lincoln, 1982: 381) EFF's research has been

7

Equipped for the Future Research Report
EFF's Approaches
To Research

E 11.
11 Page 12 13
staged, with each stage involving input from as
wide a range of people as possible, and each set of
working hypotheses has been re-presented for
review, revision, and acceptance.
In qualitative research, rigorous internal pro-cedures
are an essential requirement for trust in the
inquiry's outcomes. These are somewhat different
from the conventional requirements for rigor in
quantitative research methods - internal and exter-nal
validity, reliability, and objectivity. In qualita-tive
research the central question is of "truth value"
(Guba and Lincoln, 1982) - whether the findings
are credible to other researchers and to the sources
or subjects of the research themselves. Credibility
with the sources requires that findings be presented
to them for validation, which EFF has done. Credi-

bility with other researchers requires openness
about research methods and adoption of tech-niques
to strengthen factual accuracy, such as tri-angulation
(checking data from different sources
against each other) and cross-examination (repeat-ed
observations over time), which EFF has also
maintained.
The end purpose of practical application
means that EFF's research also has drawn on action
research traditions and more recent cooperative
inquiry (see Reason, 1994, for a summary of these
approaches). Validity in such approaches depends in
part on researchers' awareness and articulation of
the assumptions they bring to the process, while at
the same time holding themselves open to new
experiences and knowledge. (ibid., 1994: 327)

8

Equipped for the Future Research Report
EFF's research has been staged, with each stage involving input
from as wide a range of people as possible, and each set of working hypotheses
has been re-presented for review, revision, and acceptance.
12.
12 Page 13 14
HE CONSTRUCTION OF EFF REFLECTS CUTTING
edge theory on learning and teaching. In five
years of continuing work it has involved
thousands of people across the U. S. The
resulting framework has been shaped by the input
of many diverse stakeholders and program-level
development and testing.
Certain key theories and theorists played a
crucial role in the analysis and understanding of
input from the field and in the interpretation of
the task to be accomplished. There has been con-stant
interaction between practice and theory and
between data and analysis. Four conceptual threads
have shaped the EFF framework in important ways:
° a purposeful, constructivist approach to learning;
° rooting education in the context of people's lives;
° an emphasis on application, not just possession, of
skills;
° a view of adult development as transformative
rather than additive.

Purposeful View of Learning
EFF began by listening to adult learners and identi-fying
four purposes for learning that underpin the
kinds of immediate goals learners state when they
enroll in education. These purposes are consistent
with recent cognitive and socio-cultural research on
learning, which characterize it as a process of mak-ing
meaning - of organizing and interpreting expe-rience.
Mezirow, for example, describes learning as
"the process of making a new or revised interpreta-tion
of the meaning of an experience, which guides

subsequent understanding, appreciation, and
action." (Mezirow, 1990: 1) He argues that these
processes are powerfully influenced by our "habits
of expectation" - a set of assumptions that consti-tute
a frame of reference. New experiences are
assimilated and transformed by these assumptions,
which are derived from our past experiences (simi-lar
to Kegan's "orders of consciousness," described
below).
This means that learning is essentially a social
process, the ongoing process through which we
make sense of our experiences. Because our lives are
social, so are our experiences and the processes by
which we come to understand them. (Lave and
Wenger, 1991, Wenger, 1996) "Assuming that learn-ing
is fundamentally social is not denying that it
involves neurological processes, but it is placing
these processes in the social context in which we
experience them as meaningful. Learning is funda-mentally
social because we are social beings."
(Wenger, 1996: 22) This does not mean that all
learning has to happen in a group, but that our
social context shapes how we perceive, what is
important to us, and how we learn.
These concepts of learning link with research
on the social construction of knowledge (see, for
example, Scribner, 1988; Lave and Wenger, 1991).
These are accounts of the mind in action, in which
cognitive tasks are seen not as separate from daily
life but as part of its activities. "We undertake cogni-tive
tasks not merely as ends in themselves but as
means for achieving larger objectives and goals, and

9

Equipped for the Future Research Report
Concepts and Theories In EFF

T 13.
13 Page 14 15
we carry out those tasks in constant interaction with
social and material resources and constraints."
(Scribner, 1988: 1) Mental processes are integrated
with behavior: "People strive to satisfy purposes that
have meaning within their community, and, in their
activities, they use tools, symbols, and modes of
action that are culturally developed and transmit-ted."
(ibid.: 2).
EFF has drawn on these conceptions to inter-pret
learning as not simply the acquisition of skills
and knowledge but the process of assigning mean-ing
to experience and fulfilling purposes that are
important to us. EFF's conceptual framework was
distilled from ideas that have meaning for individu-als
consulted during the project - how active citi-zens,
effective workers, and involved parents
perform their roles in their particular social con-texts.
As a result, the EFF framework has focused
not on bodies of knowledge to be mastered but on
purposeful activity -" work together," "work within
the big picture," "strengthen the family system." In
an internal document for EFF, Tom Sticht focuses
on the importance of "the purposeful, dynamic view
of the person" in the conceptualization of EFF stan-dards.
He identifies as central to adult learning "the
purposeful, constructive nature of the mental
process in setting a goal, searching out the input
information, processing it by mixing it with prior
knowledge, performing an output, and monitoring
the latter as feedback for future activity." (CONS-ABE,
1996: 20) A "constructivist" approach to edu-cation
focuses on enhancing the individual's
capacity to make meaning and achieve purposes by
selecting, regulating, making decisions, and acting
upon new data.

Rooting Education in the Context of Lives
Adult education has long tried to relate education to
life experience. "Competency-based" approaches to

adult basic education, dating back to the Adult Per-formance
Level (APL) project in the 1970s, focus on
developing the skills required for the real tasks of
everyday life, rather than mastering a body of
knowledge. However, all the earlier competency-based
projects have run into problems of creating
long lists of competencies, with no a priori rationale
of how to select from them.
During the EFF planning phase, Tom Sticht
reviewed efforts to develop competency-based adult
education from the mid-1970s and identified a
number of concerns. In particular he noted:
° The "proliferation" issue: the tendency for pro-jects
to develop very long lists of competencies,
with no rationale for how many or which sub-areas
should be generated.
° The "overlap" issue: the question of the interac-tions
and similarities among the many "competen-cies" -
what underlies the lists of all the things
adults should be able to do?
° The "levels" issue: in which projects assume that
people can be assigned to levels of competency,
without any clarity on what the levels mean.
° The "development" issue: in which growth is
assumed to take place, without any clarity on how
adults come to possess knowledge and skills.
° The "who decides" issue: the danger of compiling
competencies based on constituencies other than
adult learners themselves, in contradiction to
principles of learner-centered education. [Source:
CONSABE, 1996: 22]
The "who decides" issue was the easiest to
respond to: adult learners have been consulted
throughout the EFF process, along with other con-stituencies.
The issues of proliferation and the ques-tion
of which concepts are underlying, or generative,
concerned the EFF teams throughout the develop-ment
of standards. They were well aware that many
of the K-12 standards, in trying to codify bodies of

1 0

Equipped for the Future Research Report
The EFF framework has focused not on bodies of knowledge to be mastered
but on purposeful activity -" work together," "work within the big picture,"
"strengthen the family system."
14.
14 Page 15 16
knowledge, had been unable to avoid long lists,
some very long. The history standards, for example,
have 39 main standards, 108 subheads and 526 sub-subheads.
(Gagnon, 1995: 74)
The problems of proliferation and overlap
demand a good theory. Simply collecting data about
what adults know and do could be endless. Only
theory about learning and knowing can provide the
conceptual frame that allows the central or genera-tive
items to be identified. As Glaser and Strauss say,
the work of theory in social science is (among other
things): "to be usable in practical applications -
prediction and explanation should be able to give
the practitioner understanding and some control of
situations." (Glaser and Strauss, 1967: 3) In other
words, there's nothing as practical as a good theory.
The socio-contextual theory focuses on the skills
that help people select, regulate, and act upon new
information to accomplish their particular purposes
(which in turn are socially influenced).
Similarly, theory about development would
help resolve the levels and development issues by
providing an understanding of how adults grow and
develop competence. Only if levels are not arbitrary
but signify real, transformative moments in devel-opment
(like Kegan's orders of consciousness dis-cussed
later) are they likely to be useful to practice.
These issues, and the insights from theory,
brought EFF to a different understanding of what
"context" should mean. In common Adult Basic
Education (ABE) parlance, context means domain,
the situation in which learners find themselves - at
work or in their family or community. When Sticht
initially proposed the "functional context" approach
to adult education, however, his intention was to
focus on use, not just situation. As EFF developed
role maps in an attempt to identify what adults need
to know and be able to do in their important adult
roles, it became clear that application or use, not

possession of skills, was what mattered. Context
came to be seen as the reasons people have for learn-ing,
the use they want to make of it. In this way, a
context-based approach became linked with a pur-poseful
approach to learning.

Application, Not Possession, of Skills
As EFF worked to identify what adults need to
know and be able to do as a basis for standards
development, it became clear that the application
of skills and knowledge, not their simple posses-sion,
is most important for adult education. The
work on the citizen role articulated this most clear-ly:
in the learners' consultation, people talked
about citizenship in terms of action - not simply
voting but also taking part in community life. In
the focus groups conducted during the planning
year, participants talked about citizenship as "tak-ing
action" to make a difference, about using skills
and knowledge for the common good. They
echoed the third purpose for learning - literacy as
a vehicle for independent action.
So midway through the planning year, EFF
made a crucial shift - from a focus on knowing to a
focus on doing. This shift made EFF distinct from
K-12 standards efforts, which deal with mastery of
skills and bodies of knowledge, and more like the
occupational skill standards, which focus on appli-cation
of skills in the workplace. There are theoreti-cal,
not just practical, implications here - a focus on
application rather than possession of skills and
knowledge is associated with an approach to learn-ing
as active, not passive, as constructivist rather
than accumulative, as socio-contextual rather than
autonomous. Learning then becomes a process not
of acquiring facts and skills but of enhancing one's
ability to understand one's situation, make decisions
about and act upon knowledge, aimed at transform-ing
how one views the world and acts in it.

1 1

Equipped for the Future Research Report
As EFF developed role maps in an attempt to identify what adults need to know
and be able to do in their important adult roles, it became clear that application or use,
not possession of skills, was what mattered.
15.
15 Page 16 17
Adult Development as
Transformation
Adult educators have traditionally approached adult
development as the growth of self-direction. They
assume that, as individuals develop and mature,
they become less dependent and more self-directing
(Knowles, 1980: 44). Newer research on develop-ment
conceives it as essentially a learning process.
Robert Kegan's work on psychological growth
(Kegan, 1982 and 1994) portrays development as
mastering successively more complex principles for
organizing experience - different "orders of con-sciousness."
These orders are "not merely principles
for how one thinks but for how one constructs
experience more generally, including one's thinking,
feeling and social relating." (Kegan, 1994: 32).
Kegan's orders of consciousness succeed and
transform each other from childhood through
adulthood: each is more complex, and encompasses
the prior principles. For Kegan, development does
not depend so much on learning specific skills and
knowledge as on transforming ways of knowing -
on principles for organizing and interpreting expe-rience.
EFF has drawn on these ideas about adult
development especially in conceptualizing perfor-mance.
A developmental approach to performance
means it is seen not simply as mastering more and
more knowledge and skills in a cumulative way, but
as making conceptual leaps in understanding and
viewing the world - as transformative more than
additive. This approach to development and perfor-mance
is more consistent with the purposeful view
of learning. It challenges most approaches to
accountability systems, which are based on policy-defined
outcomes and standardized tests. As Sticht
wrote,
"There appears to be a need for a developmen-tal,
theory-based means of assessing competence

in adult literacy education that bridges between
the "bottom-up" growth-oriented, developmen-tal
perspective from which teachers work, and
the "top-down" outcomes-based, statistical dif-ficulty
approach from which standardized,
normed test developers work."
(CONSABE,
1996: 15)
EFF's challenge is to create standards that
combine the purposeful elements of the human
condition with transformative adult development.
The goal is for the small steps of learning to be
clearly linked to ultimate outcomes, and for those
outcomes to be centered on activity that has mean-ing
in people's lives.
These four conceptual threads intertwine. EFF
has been an iterative process of openness to new
data and looking for theories that best explain the
data and provide a coherent framework on which to
build. As EFF developed, it confronted first one,
then another issue, and reached resolution. The
framework represents a relatively coherent concep-tual
picture, which is still evolving.
The next three sections of this report describe
the processes used and the sources of data collected
to create the framework. The assumptions guiding
the research are examined, and the credibility and
significance of the findings discussed in terms of
both theory and practice. Each of the main EFF
research activities is described:
° Section 4: Consulting with Adult Learners, which
led to the four purposes for learning (laying the
foundation for the framework).
° Section 5: Mapping the Roles of Citizen, Worker,
and Family Member (beginning the second stage,
developing content standards).
° Section 6: Identifying Key Skills and Knowledge of
Effective Role Performance (continuing the sec-ond
stage).

1 2

Equipped for the Future Research Report
EFF's challenge is to create standards
that combine the purposeful elements of the human condition
with transformative adult development.
16.
16 Page 17 18
HE 1994 CONSULTATION WITH LEARNERS ABOUT
what Goal 6 of the National Education Goals
means to them has been the foundation for
all subsequent EFF work. It reflects EFF's
consistent emphasis on the input of stakeholders -
in this case adult learners, who are often left out of
consultation. The general approach used here -
consultation, analysis of input, broad feedback from
stakeholders on the analysis - has been replicated in
subsequent phases of EFF research.
The consultation with learners was the first
step toward measurement of progress on Goal 6. As
the report on this phase says, "we understood that
without a consensus on what skills and knowledge
adults actually need to be able to participate fully
and successfully in civic and economic life we could
not determine how far we are from Goal 6 or gauge
our progress toward achieving it." (Stein, 1995: 7)

Process for Learner Consultation
The consultation with learners was part of a joint
initiative between NIFL and the National Education
Goals Panel. After discussions with representatives
of the National Adult Student Congress and the
National Coalition for Literacy, the two organiza-tions
decided to seek student input via their teachers
and tutors. Adult literacy practitioners were invited
to devote class time to a discussion of Goal 6 as a
prelude to students' writing about what the Goal
means to them. Literacy South, a training and tech-nical
assistance organization with experience in stu-dent
writing and publishing, developed a set of
guidelines for teachers and tutors to use to stimulate

discussion (Stein, 1995). Key elements were a set of
stem sentences to be completed by students:
° In my community, competing in the global economy
meansÉ
° To me, having the skills and knowledge to compete in
the global economy meansÉ
° To me, exercising the rights and responsibilities of
citizenship meansÉ
° To exercise the rights and responsibilities of citizen-ship
you have to be able toÉ
Following discussions, learners were asked to
respond in their own words and submit their writ-ing
to NIFL. The invitation to participate was issued
in January 1994 in an open letter to adult learners,
their teachers, tutors, and program directors. Mem-ber
organizations of the National Coalition for Lit-eracy
actively participated in getting materials to
their constituents. 2 About 6,000 invitation packets
were distributed by NIFL and Coalition for Literacy
members. The goal was to get a broad response
from across the country, from different regions and
types of programs.
By the end of March 1994, NIFL had received
more than 1,500 responses from students in 151
adult literacy programs in 34 states and Puerto Rico.
These represent a substantial data source: the average
learner response was two paragraphs, some were
much longer, and the data set fills a 3-foot file draw-er.
No attempt was made to draw a representative
national sample either of programs or of learners:
the responses were self-selected. The overall response
rate is impossible to calculate given the dispersed
method of recruitment. 3 However, the geographical

1 3

Equipped for the Future Research Report
The Foundation: Purposes for Learning

T 17.
17 Page 18 19
distribution and breadth of programs and learners
represented is wide. It includes the full spectrum of
types of literacy programs: English as a Second Lan-guage
(ESL), literacy, adult secondary education, and
adult high school programs; family literacy, work-place,
and prison programs; programs based in com-munity
organizations, volunteer groups, community
colleges, vocational and public schools. The respons-es
came from metropolitan, urban, suburban, and
rural areas, and from a wide range of ages and ethnic
groups.
NIFL did not intend a quantitative analysis of
responses - counting how many learners had said
what, and deducing conclusions about the charac-teristics
of the entire learner population. If it had,
there would have been grounds for concern about
the representativeness of the sample. In qualitative
research, however, analysis is essentially theoriz-ing -
discovering patterns and relationships among
the data, and testing these against each other and
against expectations from prior research and theory
(Goetz and LeCompte, 1984; Guba and Lincoln,

1982). Conceptual accuracy is the goal. EFF fol-lowed
rigorous analytical procedures and credibility
checks throughout.

Analysis of Learner Responses
Analysis of the learner responses followed typical
procedures for qualitative research (e. g. Goetz &
LeCompte, 1984). As the texts from adult learners
came in, each was assigned a unique code with
region, program, and learner identifiers. The first
step in the analysis was to develop a coding frame.
Sometimes qualitative researchers design a coding
frame in advance of looking at the data, based on
theory. More often, as here, the coding frame
emerges from the data itself. The eight-member
analysis team 4 spent several days reading and dis-cussing
a broad cross-section of learner responses in
order to identify themes that appeared robust
enough to be used as categories for coding. 5 The
result was four main categories, each with sub-cate-gories
(see Figure 1).
A smaller team then coded all learner respons-1

4

Equipped for the Future Research Report
SUB-CATEGORY
Daily life/ tasks
Feel better about self
Future orientation
Religious practice
Understanding the world (external)
Protection/ vulnerability
Able to communicate

Set a good example for children
Help to improve family circumstances

Help children with schooling
Read to children
Help children with moral/ intellectual development

SUB-CATEGORY
Get a job/ better job
Keep up with change/ technology
Keep jobs in America
Language and cultural skills
Economic awareness

Job-related literacy tasks
Roles and Responsibilities of Citizenship

Be an informed citizen
Participate in political process
Understand and strive for rights
Understand and fulfill responsibilities
Gain citizenship
Contribute to community

CATEGORY
Personal
Development

Family/
Parenting

CATEGORY
Job/ Compete
in Global
Economy

Roles and
Responsibilities
of Citizenship

Figure 1. CODING CATEGORIES

The eight-member analysis team spent several days reading
and discussing a broad cross-section of learner responses in order to identify themes
that appeared robust enough to be used as categories for coding.
18.
18 Page 19 20
es and entered them on Ethnograph (computer soft-ware
designed for analysis of qualitative data). 6 Each
coder was assigned two of the four categories, so
each category was coded by two people, a step
included to increase reliability. After the first week of
coding, the team tested the consistency of interpreta-tion
of the definitions. Team members met weekly to
discuss issues, resolve differences and problems, and
reach agreement on how to handle data consistently.
They made every attempt to ensure consistency
between coders and agreement on the interpretation
of the data.
Following the initial coding, the team began
synthesizing the data. The coders prepared ten-page
working papers that summarized and illustrated
findings, sub-category by sub-category, for each of
the two primary categories they had coded. This
provided two different coders' interpretations for
each category, again increasing reliability. Finally,
the entire analysis team met to discuss all eight sum-mary
reports, compare findings across categories,
and draw conclusions.

Four Purposes for Learning
The synthesis step that followed was a crucial one
for Equipped for the Future. The original coding
categories were reassuringly familiar to adult literacy
practitioners: they are the kinds of reasons adult
learners often give for enrolling in programs. 7 As
with all such lists of individual learning goals, they
could have been elaborated further into ever more
specific and particular categories. Instead, a different
set emerged that crosscut and linked with the origi-nal
codes, but created a new way of looking at pur-poses
of learning. The EFF synthesis relates to
learners' specific goals reflected in the original cod-ing
guide, but goes beyond individual and particular
learner goals to identify four underlying purposes
for adult learning (Figure 2).

These were conceived as "fundamental pur-poses
that express the social and cultural meaning
or significance" of the more specific individual goals
(Stein, 1995: 9). They are the ultimate goals of peo-ple
"engaged in defining themselves as competent
actors in the world" (ibid.), and they drive learning
across the different contexts of adult life. As such,
they are consistent with the theories of learning and
adult development described in Section 3, with their
emphasis on social context, meaning, and action.
Learning for access and orientation includes
not only physical or geographic orientation - read-ing
maps and signs - but also psychological or
social orientation - knowing what is going on in the
world, understanding institutions that have an
impact on one's life, getting needed information.
This purpose underlies many of the specific goals in
the coding frame across all four coding categories -
for example, understanding the world, helping chil-dren
with schooling, getting a job, gaining economic
awareness, and being an informed citizen.

1 5

Equipped for the Future Research Report
In order to compete in the global economy
and exercise the rights and responsibilities
of citizenship, adults need the skills and
knowledge:
° To have access to information and orient
themselves in the world;

° To give voice to their ideas and opinions
and to have the confidence that their voice
will be heard and taken into account;

° To solve problems and make decisions
on their own, acting independently as a
parent, citizen and worker, for the good of
their families, their communities, and their
nation;

° To be able to keep on learning in order to
keep up with a rapidly changing world.

[Source: Stein, 1995: 4]

Figure 2. FOUR PURPOSES FOR LEARNING
In the synthesis a different set emerged
that crosscut and linked with the original codes,
but created a new way of looking at purposes of learning.
19.
19 Page 20 21
Learning for voice embraces all aspects of
communication - written and oral - needed to pre-sent
oneself to the world. It goes beyond communi-cation
skills to the reasons for communicating: to
speak and be heard. The writings about citizenship
offered an important arena for voice, but it was also
important to adults in other aspects of their lives -
to communicate with their children's teachers, to
exchange ideas at work, to speak up in their com-munity.

Learning for independent action includes the
dual elements of independence and action. Many
adults who feel their literacy skills are limited
depend on others for help with reading and writing.
In writings that pointed to this purpose, learners
expressed their desire to be able to act for them-selves,
to make informed decisions, and not have to
rely on others to tell them what to do. Independence
emerged most strongly in the personal development
categories, but learners' responses stressed indepen-dent
action in all aspects of life - supporting their
families, achieving economic self-sufficiency, and
fulfilling their responsibilities in their communities.
Learning as a bridge to the future reflected
learners' sense that the world is changing. A prime
purpose for learning is to be ready for the changes,
to learn how to learn, and to prepare oneself for life-long
learning. Particularly at work, keeping up with
change is a necessity, but in personal and family
development and citizenship, learners saw them-selves
in rapid social transformation. Keeping a job,
adjusting to technological change, and improving
family circumstances were all reasons to continue
learning.
The purposes are essentially about lifelong
learning for everyone, not just literacy students.
They underpin a person's particular reasons for
learning at any point of time. They are simple, pow-erful
ideas. We might wonder why we are only now

identifying them. In part, the answer lies in the sheer
volume of the data, which allowed underlying pur-poses
to emerge from the particulars: usually learner
goals are treated individually, not in aggregate form.
In part, it may lie in the context of asking the ques-tion.
When we ask one learner at a time in the con-text
of an education program, they give concrete
and particular goals - to read better, to help their
children in school, to get their GED. When we ask
learners in the context of their roles what they need
to be most effective as a parent, worker, or family
member, they give different kinds of answers.

Feedback and Validation of the Purposes
NIFL published a report on the learner consultation
in July 1995, and invited responses and comments
(Stein, 1995). Since then, the purposes for learning
have been widely presented to stakeholders in con-ferences
and workshops. About 20,000 copies of the
published report have been distributed, and a steady
demand for it continues. It is also on NIFL's website.
The purposes for learning were further explored
and validated in the next EFF phase, during 1995-
96, by eight planning projects that held focus groups
and discussions with a wide range of learners, prac-titioners,
and stakeholders. The credibility of the
purposes for learning was enhanced by their subse-quent
presentation to learners and other stakehold-ers.
The purposes have proved remarkably resilient
and resonant.
The work of the planning projects is described
in more detail in Section 5. They approached the
task of testing and validating the four purposes in
various ways. Some projects asked focus group par-ticipants
to comment directly on the purposes.
Others analyzed the data from focus groups within
the frame of the four purposes. Some tested the pur-poses
against their focus group data to see if they
were congruent. Whatever the approach taken, the

1 6

Equipped for the Future Research Report
The purposes are essentially about lifelong learning
for everyone, not just literacy students. They underpin a person's
particular reasons for learning at any point of time.
20.
20 Page 21 22
planning projects found that the four purposes were
consistent with their new data.

The Importance of the Purposes
for Learning
Accountability systems require a concept of perfor-mance -
desired results - in order to measure
progress toward them. When framed too close to the
individual learner, learner goals are too particular to
connect with system accountability, which is framed
at societal level. Conversely, the kinds of high-level
goals that accountability systems often construct -
get the GED, enter vocational training, get a job -
do not necessarily reflect the goals of all learners.
There may be a mismatch between individual and
system definitions of performance. Only if broad,
system-level goals are customer-driven will account-ability
systems work effectively, and produce a real

difference in learners' lives. EFF's four purposes for
learning provide a link, a bridge, between account-ability
systems and learners. The purposes for learn-ing
are a foundation on which the rest of the
framework has been constructed.
The four purposes for lifelong learning were
the first step, but on their own they are not enough
to change the adult basic education system. They
describe why people learn, the purposeful side of
learning, but not what needs to be learned. EFF
needed to take the next step to describing the "what"
by identifying the broad literacy skills needed to be
effective in the primary adult roles. The next stages
of EFF built on the purposes for learning by creating
"role maps" for the adult roles of worker, citizen/
community member, and parent/ family member
through research and validation with the field and
other stakeholders.

1 7

Equipped for the Future Research Report
EFF's four purposes for learning provide a link, a bridge,
between accountability systems and learners. The purposes for learning are a foundation
on which the rest of the framework has been constructed.
21.
21 Page 22 23
OST STANDARDS PROJECTS START
by mapping the terrain and
developing a careful review of
practice in their area, includ-ing
(for skills standards) consultation with "subject
matter experts." In 1995, following the learner con-sultation
report, Equipped for the Future set out to
map the terrain of "what adults need to know and
be able to do to fulfill their roles as parents, citizens,
and workers" (NIFL, 1995c: 35950). These three are
not the only adult roles, but are key ones for public
policy. 8 In the first stage of the task, eight planning
projects carried out a series of focus groups with
stakeholders designed to describe what adults in
each role know and are able to do. In successive
phases the focus group results were synthesized into
draft role maps, presented for extensive feedback
from a wide range of role practitioners, revised, and
published. Rather than starting with what children
learn in school, and conceiving adult education as
an attempt to remediate past gaps in knowledge and
skill, role maps start from adult life.
In the course of this work, the important shift
was made from a focus simply on skills and knowl-edge
to one on action, application, and use of skills
and knowledge (noted above in Section 3). The
emphasis on what adults do, on broad areas of
responsibility and key activities, emerged most
strongly in the course of the focus groups convened
by the Civic Participation Project (CPP) in New
England and southern Appalachia. By April 1996 it
was becoming clear that these focus groups were

talking about citizenship in different ways from the
conventional "civic education" approaches. Grass-roots
community activists and adult learners in
particular were talking about citizenship as "taking
action" to make a difference. These actions were not
just the usual ones of voting and participating in
the democratic process, but also much smaller, local
steps - helping a neighbor, taking part in commu-nity
clean-up, volunteering in schools. Citizenship
at its core was not simply a matter of having skills
and knowledge, but about using them for the com-mon
good.
This perspective was consistent with construc-tivist
theory, in which the central aspect of human
activity is seen as that of shaping (constructing)
experience. The responses reinforced and strength-ened
that perspective. The role maps did not simply
list everything adults do in daily life (the result of so
many competency-based initiatives). People were
asked to identify the most important - the central,
defining activities in terms of meaning and value -
so that a concept could be built of the critical role
functions that cross geographical, race, class, and
gender divides. This is what has given the role maps
their resilience.
As with the four purposes for learning, the
process of creating role maps was a naturalistic
inquiry that involved collecting new data, analyzing
and synthesizing it, and re-presenting it to stake-holders
for refinement and validation. Each stage
will be described in the rest of Section 5: collection
of new data from which initial draft role maps were

1 8

Equipped for the Future Research Report
Role Maps for Effective Citizens, Workers, Parents

M 22.
22 Page 23 24
constructed, their refinement and validation
through structured feedback and inquiry processes,
revision of the role maps, and initial drafting of
"role indicators," which provide a basis for develop-ing
performance measures.

Raw Materials for the Role Maps
The overall charges to the eight planning projects
funded by NIFL in 1995-96 were: to explore and test
the four purposes; examine what adults need to
know and do as workers, citizens, and parents; and
begin to engage a wider array of stakeholders in the
conversation about developing content standards
for the field. The focus groups conducted by each
project provided the data from which the draft role
maps were constructed.
The eight projects worked in a total of 18
states (see Figure 3). Five projects worked in only
one state, the other three in multiple states. Some
projects addressed all three roles, some only one
role. While there were differences in how each of the

projects interpreted its task and carried out its work,
there was a common structure. All eight projects
convened Working Groups that brought together
various stakeholders - adult education and employ-ment
training, business and community organiza-tions.
All convened focus groups to examine key
questions about what adults need to know and be
able to do to fulfill their roles. One project also con-vened
inquiry projects, each with an ABE or ESL
teacher and students.
A total of 1,109 participants were involved in
114 focus groups. While the largest single group of
participants was adult learners, the focus groups
also involved adult education practitioners and a
wide variety of others with a stake in adult educa-tion -
civic and community activists, elected and
government officials, employers and employees,
clergy, media representatives, social services work-ers,
and teachers in a variety of program settings.
Demographic data are not available on all focus
groups, but in the 74 groups for which data were

1 9

Equipped for the Future Research Report
Adult Numeracy Practitioners' Network 7 states:
(numeracy aspects only) IL, VA, OR, OH, NH, VT, RI

CONSABE (Content Standards for San Diego, CA
Adult Basic Education)

Maine Statewide Standards Maine
Development

Minneapolis Public Schools Minneapolis, MN
National Center for 5 states:
Family Literacy AZ, WI, LA, KY, TN

North Carolina Literacy Resource Center North Carolina
Philadelphia Mayor's Commission Philadelphia, PA
on Literacy

Southern Appalachian and 8 states:
New England Civic Participation Project KY, VA, TN, VT, MA, RI, NH, ME

ROLE PROJECT SITES Worker Parent Citizen
Figure 3. 1995-96 PLANNING PROJECTS

People were asked to identify the most important - the central, defining activities in terms
of meaning and value - so that a concept could be built of the critical role functions that cross
geographical, race, class, and gender divides. This is what has given the role maps their resilience.
23.
23 Page 24 25
reported, the majority of participants was female
(sometimes a significant majority, as in the parent/
family role groups), and the age range was from
teens to 60s, with a majority usually in the 20s and
30s. The ethnic/ racial mix varied considerably, from
majority white in southern states and Maine to
majority African American and Asian in Minneapo-lis.
Most of the adult learner focus groups were
more racially diverse than the stakeholder groups
(see Appendix A).

Developing Draft Role Maps
Each planning group presented an end-of-year
report on its findings. Each project had set out on a
somewhat different path, convening different kinds
of groups and asking different questions, but each
reported their data in a common format, defined by
NIFL as follows: "Our goal is to define as fully, as
concretely, as specifically as possible, what any/ every
adult needs to know and be able to do to fulfill their
responsibilities in the three key roles related to Goal
6: parent/ family member, citizen, worker." (Stein,
internal document, April 18, 1996) For each role,
there were four research questions:
1. What have we learned about what adults consid-er
to be the broad areas of responsibility for their
roles?
2. For each broad area of responsibility, what key
activities are important/ even necessary parts of
being effective in that role?
3. What have we learned about what adults need to
know and be able to do in order to carry out
those responsibilities?
4. How does the research help us better understand
the four purposes for learning? How should they
be defined and refined? [summarized from Stein,
internal document, April 18, 1996)
Each project reported on its data using this
structure for each role. The eight separate reports

were then synthesized across roles. 9 The syntheses
drew on all the data for that role, looking for com-mon
patterns reported by the projects. Although
different "conceptual maps" were developed by the
projects on the basis of their own data sets, there
were many commonalities in reported data. A "pic-ture"
of the three key adult roles emerged from the
syntheses, including broad areas of responsibility,
key activities, the skills and knowledge needed to
accomplish the activities, and the interrelationship
between them, moving toward a common concep-tual
frame for each role.
The language chosen for the syntheses came
from the data and reflected the purposeful and
active depiction of the roles -" taking action," for
example. The data were consistent with seeing
learners as purposeful as well as cognitive beings -
as Kegan puts it, "organizers of their experience." It
was hoped the framework would be less a "founda-tion"
(something static and unchanging) and more
a "core" (a dynamic source of energy and fusion). In
keeping with the guidelines the grantees had devel-oped,
the language also needed to be clear, simple,
and understandable to learners, practitioners, and
the general public.
The synthesis for each role was completed in
fall 1996, and the initial draft role maps were pub-lished
in February 1997. (see Stein, 1997) These
draft role maps were not fully elaborated and did
not include skills and knowledge. Each defined the
"key purpose" that illustrates the central aim of the
role, "broad areas of responsibility" that are the crit-ical
functions an adult performs to achieve the role's
key purpose, and "key activities" through which the
role is performed. In keeping with the naturalistic
approach to inquiry, the credibility of the draft role
maps was not assumed but tested through several
iterations of consultation with stakeholders and
other informants.

2 0

Equipped for the Future Research Report
A "picture" of the three key adult roles emerged from the syntheses, including broad areas of
responsibility, key activities, the skills and knowledge needed to accomplish the activities, and the
interrelationship between them, moving toward a common conceptual frame for each role.
24.
24 Page 25 26
Structured Feedback to Revise
and Validate the Role Maps
Three consortia were funded by NIFL in fall 1996 to
take responsibility for the next phase of work: the
refinement and validation of the role maps as a step
toward developing content standards for adult edu-cation.
Although each role was approached sepa-rately,
the consortia used a common process for
structured feedback across all roles. Each consor-tium
convened a national advisory group represent-ing
stakeholders, and the worker role consortium
also convened state working groups in five states.
The consortia were as follows:
° Citizen Role: Led by Center for Literacy Studies,
The University of Tennessee, Knoxville, in partner-ship
with Mayor's Commission on Literacy,
Philadelphia, and New England Literacy Resource
Center.
° Parent/ Family Member Role: Led by National
Center for Family Literacy, in partnership with
Arizona Department of Education, Virginia
Department of Education, Houston Community
College System, and Wisconsin Department of
Public Instruction.
° Worker Role: Led by The Center for Adult Literacy
and Learning, University of Maine, in partnership
with North Carolina Literacy Resource Center,
Ohio Literacy Resource Center, Vermont Adult
Learning, Virginia Department of Education, and
Adult Numeracy Network.
These consortia carried out an extensive feed-back
process on the draft role maps over some five
months, from January through May, 1997. Each
consortium recruited effective role performers and a
variety of stakeholders to comment on and revise its
draft role map. The worker role process involved
industries and unions in five states, the citizen role
included people active in a wide range of civic orga-nizations
in 13 states, and the family role recruited

parents and family members through a variety of
groups in 9 states. A total of 864 participants from
18 states took part. Participants were geographically
spread out, and diverse in terms of demographics,
background, and experiences (see Appendix B).
Performance Consulting Inc. (PCI), a techni-cal
assistance team experienced in work with occu-pational
skills standards, SCANS and O* NET,
trained facilitators for the feedback sessions to use
common process guidelines across all three roles:
° In each session, participants were first introduced
to the EFF project and provided information
about the work to date, including the purposes for
learning and the other draft role maps.
° They were then invited to comment in detail and
suggest revisions to the draft role map, based on
their own experience and knowledge. Small groups
worked on particular parts of the role map and
reported back to the whole group with any revi-sions
they proposed. These were then discussed by
the group as a whole.
° At the end of each session facilitators attempted to
have the group reach a consensus on revisions
to the role map. If there was disagreement and
consensus could not be reached, the issue was pre-sented
to the next feedback session for its consid-eration.

The credibility of the process of developing
and refining the role maps is affected by the recruit-ment
of participants for feedback sessions. Two
aspects in particular are important: the identifica-tion
of "high performers" and the diversity of par-ticipants.

High Performers: The identification of "high
performers" reveals EFF's links with the occupation-al
skills standards, which commonly spoke of high
performance workers. This proved challenging for
the consortia to put into practice. High performers
in a particular role were defined as people who:

2 1

Equipped for the Future Research Report
In keeping with the naturalistic approach to inquiry,
the credibility of the draft role maps was not assumed but tested through
several iterations of consultation with stakeholders and other informants.
25.
25 Page 26 27
5. display commitment to quality in role perfor-mance;
6. actively participate in the role and demonstrate a
higher than average level of achievement in the
role;
7. show initiative in role performance and have
earned the respect of others through their role
performance.
The concept was least problematic for the
worker role. Each of the five participating states
convened an interagency working group that was
asked to define criteria for high performance work-ers,
to ensure that the criteria were appropriate for
the particular industry targeted. The working
groups wanted to broaden the concept of "high per-formance"
beyond the emphasis on productivity,
which it commonly denotes, to include interperson-al
and leadership qualities. For example, one work-ing
group came up with the following suggestions to
guide recruitment:
"Ask front-line workers, management, and customers
the following:
° Who do you go to for help?
° Who do you feel comfortable with when you work?
° Who meets or exceeds the workplace technical stan-dards?

° Who gets good customer comment cards?
° Who excels at performance reviews?
° Who has moved up in the company, or has the
potential to move up?"
(High Performance Worker
Criteria, February, 1997)
For the other two roles, there was no precedent
for identifying "high performers," and the concept
itself was less clear - what would a "high perform-ing
citizen" be, or a "high performing parent?" The
concept itself seems less appropriate for these roles,
where there is such a broad spectrum of possible
ways to perform, with enormous cultural, class, eth-nic,
and geographic differences. The research

process demanded that differences be acknowledged
while gaining input that was in some way indicative
of effective role performance.
The citizen role consortium focused its
recruitment on the concept of "active in the com-munity,"
with as wide a range as possible of contexts
and levels. It identified participants through the
national civic organizations represented on the
national Working Group, state networks developed
in the earlier EFF phases, and adult education net-works.

The parent role consortium used the term
"effective parents," which was defined in different
ways by different people. Conveners of feedback
sessions were asked to consult with community
leaders to recruit people who from their own expe-rience
could contribute to discussions about effec-tive
parents. Some participants were identified by
school principals and counselors, ministers, com-munity
service workers, and college teachers. Others
were participants in parenting and family literacy
classes. Some were self-selected. The consistent cri-terion
for "effective parents" was that someone
(themselves or others) had defined them as such.
Diversity of participation: The other recruit-ment
issue concerns whether participants brought
to the discussion a wide range of social and cultural
contexts, perspectives and knowledge bases. For all
three roles, deliberate efforts were made to recruit
from as broad a range as possible, within the con-straints
of time and resources.
For the worker role, the complexity and range
of work contexts made the task of validating the role
map particularly challenging. Staff recognized they
could not feasibly convene structured feedback from
all industrial sectors and work situations, but
instead aimed for diversity - including both manu-facturing
and service sectors, and large, medium,
and small enterprises. The aim was not to replicate

2 2

Equipped for the Future Research Report
For all three roles, deliberate efforts
were made to recruit from as broad a range as possible,
within the constraints of time and resources.
26.
26 Page 27 28
the occupational skills standards work on specific
job clusters, but to identify the key underlying char-acteristics
of workers common to many different
work contexts.
Five states participated, each targeting a par-ticular
industry and led by a partner organization
(see Figure 4). Each convened a Working Group with
representatives of the targeted industries, adult edu-cation,
employment training, economic develop-ment,
and related agencies. These Working Group
members played an important role in helping to

set up structured feedback sessions and recruit par-ticipants.
A total of 371 individuals took part in 28 feed-back
sessions on the worker role map. Almost all
were from business and industry, with a few from
education and training organizations and communi-ty
organizations (see Figure 5). Participants included
employees at all levels, from janitors to mid-level
managers. They worked in a range of sectors and
company sizes, from "Mom and Pop" grocery stores
to large multi-nationals.

2 3

Equipped for the Future Research Report
VIRGINIA
Food Industry Đ including Nabisco, Interbake
Foods, National Fruit Product Co., WLR Foods,
AFL-CIO and Bakery, Confectionery, and Tobacco
Workers International Union.

NORTH CAROLINA
Metals Manufacturing Đ including Piedmont Triad
Center for Advanced Manufacturing, AMP, Inc.,
Burckhardt American, Newman Machine Co.,
National Institute for Working Skills, and AFL-CIO.

OHIO
Heavy Metal Industry Đ including Ford, Ohio
Stamping and Machine, American Federation of
State, County and Municipal Employees, Ohio
State Building and Construction Trades Council
and AFL-CIO.

VERMONT
Health Care Industry Đ including Lamoille Mental
Health, Southern Vermont Home Health, Brattleboro
Memorial Hospital, Rutland Regional Medical
Center, Copley Health Systems, Fletcher Allen
Health Care and AFL-CIO.

MAINE
Retail Đ Including National Retail Federation,
Aroostook Center Mall, Hannaford Bros. Co.,
American Pulpwood Association and AFL-CIO.

Figure 4. STATES TAKING PART IN WORKER
ROLE MAP FEEDBACK
Figure 5. SUMMARY OF WORKER ROLE
STRUCTURED FEEDBACK PARTICIPANTS 10

REGIONS
Northeast 218 (59%)

Southeast 57 (15%)
Midwest 96 (26%)
TOTAL PARTICIPANTS 371

ORGANIZATIONAL AFFILIATIONS
Business/ Industry 208 (56%)
Union 46 (12%)
Voluntary/ Community/ Religious 21 (6%)
Education 117 (32%)
Government 14 (4%)

DEMOGRAPHICS
Male 101 (27%)
Female 217 (58%)
White 302 (81%)
African American 52 (14%)
Hispanic 2 (< 1%)
Other 5 (1%)

Age: 18-25 23 (6%)
26-35 80 (22%)
36-49 184 (50%)
50+ 73 (20%)

Participants in worker role map feedback sessions included employees at all levels,
from janitors to mid-level managers. They worked in a range of sectors and company sizes,
from "Mom and Pop" grocery stores to large multi-nationals.
27.
27 Page 28 29
For the citizen role a total of 25 structured
feedback sessions were held in 13 states, involving
257 participants (see Figure 6). Of the 242 who com-pleted
registration form details, a large majority was
female (70%). There was more diversity in terms of
race and ethnicity: 59% were white, 18% were His-panic
and 12% African American. Almost all were

U. S. citizens, but 37 did not speak English as their
first language.
Participants in the citizen role structured feed-back
were active in many kinds of civic organiza-tions,
and brought experience in a wide range of
civic participation activities. The largest single
grouping was of local community based organiza-tions.
Others were active in state and local govern-ment
(elected and appointed positions), religious
organizations, unions and businesses, educational
and academic organizations, and a variety of local
chapters of national organizations. Given that the
target population of people who are "active in their
community" is undefined, the representativeness of
these participants cannot be judged, but they cover a
range of types of involvement in community and
civic affairs.
For the parent role, 17 feedback sessions were
held in 9 states, involving 236 individuals (see Figure
7).
Participants represented all economic levels, from
welfare recipients to high-income parents. The par-ents
who worked outside the home had a wide vari-ety
of jobs: farmers, teachers, nurses, service workers,
managers in small and large businesses, company
directors, and a CEO of an international company.
Some were retired (including a former state legisla-tor).
They had a range of values, from conservative
religious to more liberal. Their education ranged
from those working to get their GED to a sprinkling
of Ph. Ds. They were affiliated with a wide variety of
organizations - government, education, libraries,
community and voluntary organizations, religious
organizations, youth groups, business and industry,
and health care. Most (198) of the participants had
English as their first language; 38 did not. Two of the
sessions were conducted in Spanish. Two groups
were conducted on Native American reservations.
In addition to these structured feedback ses-sions,
the parent role map was presented to other

2 4

Equipped for the Future Research Report
Figure 6. SUMMARY OF CITIZEN ROLE
STRUCTURED FEEDBACK PARTICIPANTS 10

REGIONS
Northeast 88 (36%)

South 115 (47%)
West/ Midwest 39 (16%)
COMPLETED REGISTRATION 242

ORGANIZATIONAL AFFILIATIONS
Community Groups 75 (33%)
Volunteer Organizations 28 (12%)
State/ Local Governments 39 (17%)
Religious Organizations 15 (7%)
Business/ Industry 11 (5%)
Labor Unions 3 (1%)
Education/ Academic 66 (29%)
Foundations 6 (3%)
Other 39 (17%)

DEMOGRAPHICS
Male 66 (27%)
Female 170 (70%)
White 143 (59%)
African American 30 (12%)
Hispanic 43 (18%)
Asian/ Pacific Islands 6 (2%)
Other 7 (3%)

Age: 18-25 6 (2%)
26-35 25 (10%)
36-49 116 (48%)
50+ 77 (32%)

Participants in the citizen role structured feedback brought experience
in a wide range of civic participation activities.
28.
28 Page 29 30
groups for feedback. In February 1997 a national
meeting of 200 Even Start coordinators from 32
states gave feedback on an early draft of the role
map. In summer 1997, an advanced training session
for family literacy practitioners held by the National
Center on Family Literacy devoted a session, involv-ing
17 teachers and administrators from 8 states (a
majority from Kentucky and Tennessee), to feed-back
on the role map. Finally, the National Parent-Teacher
Association staff and volunteers reviewed
and made suggestions for revising the role map.
Recruitment for the feedback sessions across all
three roles deliberately sought a broad range of
input, and achieved that goal. Midway through the
feedback process, a check was made of the demo-graphic
composition of the participants, and
remaining sessions tried to target any imbalances.
The family member role, for example, added a group
of non-parents. The participants were not a random
sample, nor do they represent the demographics of
the entire adult population. But they do represent a
very wide range of perspectives on role performance.

The Inquiry Process
In addition to the structured feedback process, the
citizen role consortium initiated a series of inquiry
projects to test the draft role map in adult education
classroom settings. These involved 36 teachers, each
with a group of students, in 23 program sites and
nine states (see Figure 8). The inquiry sites were both
urban and rural, and represented a range
of types of program settings - ESL, ABE-1,
GED preparation, adult high school,
family literacy, and prison programs.
They were based in different organiza-tions -
a library, a Private Industry
Council, several community-based orga-nizations -
as well as adult education
programs.

2 5

Equipped for the Future Research Report
Figure 7. SUMMARY OF PARENT ROLE
STRUCTURED FEEDBACK PARTICIPANTS 10

REGIONS
East/ Southeast 56 (24%)

South 98 (42%)
Midwest 38 (16%)
Southwest 44 (19%)
TOTAL PARTICIPANTS 236

ORGANIZATIONAL AFFILIATIONS
Parent Education 22 (9%)
ABE 62 (26%)
Family Literacy 44 (19%)
Parents From the Community 108 (46%)

DEMOGRAPHICS
Male 26 (11%)
Female 210 (89%)
White 117 (50%)
African American 62 (26%)
Hispanic 35 (15%)
Asian/ Pacific Islands 4 (2%)
Other (Native American) 18 (8%)

Age: 18-25 24 (10%)
26-35 73 (31%)
36-49 98 (42%)
50+ 41 (17%)

New England 5 2 3 3 3
Southeast 7 5 2 6 1
Pennsylvania 6 0 6 5 1
California 3 0 3 3 1
Texas 2 1 1 2 2
TOTAL 23 9 14 19 8

REGION TOTAL RURAL URBAN ABE/ ASE ESOL 11
Figure 8. INQUIRY PROJECT SUMMARY

Participants in the parent role feedback sessions represented all economic levels. The parents who
worked outside the home had a wide variety of jobs: farmers, teachers, nurses, service workers,
managers in small and large businesses, company directors, and a CEO of an international company.
29.
29 Page 30 31
Each teacher chose to focus instruction on a
broad area of responsibility in the citizen role. All
kept logs of the work and submitted reports, includ-ing
samples of student work. The inquiry projects
enabled the citizen role consortium not only to gain
feedback from adult learners on the draft role map,
but also to begin to explore how the role maps
might be used in the classroom. Because the projects
took place over several months, they allowed more
in-depth exploration of the meaning of concepts
such as "work together." Teachers investigated how
these might be addressed as part of literacy educa-tion.
They also gathered information about the skills
and knowledge associated with the key activities.
This information was drawn on in the later stage of
identifying skills and knowledge (see Section 6).

Revising the Role Maps
Each role map went through a number of revisions
based on the structured feedback and inquiry pro-jects,
until project staff felt there was broad consen-sus.
Often at this point, participants in new feedback
sessions would agree on the concepts but suggest
minor wording changes. Both the worker and citi-zen
role maps were revised twice during the feed-back
process, based on analysis of the proposed
revisions, with the revised map presented to the
next feedback sessions. The maps were revised again
at the end of the process. Because of contract delays,
the parent/ family member role consortium started
the feedback sessions a month later than the others,
and revised the draft role map after each session.
For each role map, the revisions were carried
out by a small work group of staff and associates
who were experienced in adult basic education and
employment training. They reviewed the proposed
revisions, consolidated similar suggestions, and
made judgments about merging others, using both
the proposed revisions and recordings of discussion

about them to clarify intent. Where the data clearly
identified real differences, the work group listed the
unresolved issues and asked the next round of feed-back
sessions to address the disagreements and sug-gest
resolutions. All such issues were resolved by the
end of the feedback process.
The most extensive changes were made in the
parent role map, which was redefined as a "family
member" role map. This change came from partici-pants
in early feedback sessions, who wanted the
role map to reflect a broader range of family respon-sibilities.
One session created definitions of "family"
and "parenting," which were then taken to subse-quent
feedback sessions and broadened some-what -
to include other children in one's care, for
example (see Figure 9).

For all three roles, there were some changes in
the "purpose" statement and in the "broad areas of
responsibility," but the main changes on the role
maps as a result of the structured feedback were at
the level of key activities. (For an example from the
citizen role map, see Figure 10.)
Such changes were partly about wording, and
reflected attempts by the feedback groups to ensure
that the wording conveyed their understanding of
the meaning more accurately. But in part, the
changes reflected refinement and expansion or
focusing of the activities themselves. By asking par-ticipants
to help construct a role map that reflects
their experience of what it means to be a citizen,
worker, or family member, EFF was addressing

2 6

Equipped for the Future Research Report
Family: A group of people who have common
values and common bonds, living under the
same roof. Loving and caring for one another.
All families are different!

Figure 9. DEFINITIONS OF "FAMILY"
AND "PARENTING"

Each role map went through a number of revisions based on
structured feedback and inquiry projects,
until project staff felt there was broad consensus.
30.
30 Page 31 32
meaning and values. Citizenship is not just about
voting in elections, for example, but about taking
action in many ways to make a positive difference in
the world. Parents are seen as creating a vision for
the family, and promoting values, ethics, and cultur-al
heritage. Workers not only do the work but pur-sue
work activities that bring personal satisfaction
and meaning to them. The issue of values and
meaning is integral to the role maps; they are not
decontextualized lists of skills.
Disagreements about values arose and needed
to be resolved. For example, in the family role feed-back,
one group made the proposal that a key activi-ty
should be "take children to church." While some
parents felt strongly that this should be included,
others did not agree - they said that they do not
belong to a church but are spiritual. This issue was
taken to other groups, who recommended a strong
reference to spirituality but not to church atten-dance.
Participants in the feedback sessions repre-sented
diverse social contexts, with often conflicting

value systems. However, by keeping the role maps
broad and general, EFF attempted to construct a
framework on which people with divergent value
systems could agree.
As a result of the structured feedback process,
there can be some confidence that the broad areas of
responsibility and the activities in the role maps rep-resent
a credible portrait of the three adult roles,
distilled from the experience of a broad sector of the
population.

Role Indicators
To move toward standards it is not enough to map
broad areas of responsibility and key activities. We
also need to know what successful performance of
these activities looks like in order to identify the
skills and knowledge needed to do them. "Role indi-cators"
describe successful performance of key activ-ities,
and so provide an important link between
activities and skills. In the later stages of role map
revision, during April and May 1997, participants in

2 7

Equipped for the Future Research Report
Forming and
expressing opinions
and ideas

Develop a sense of self
in relation to the world

Communicate so that
others understand

Form and express
opinions and ideas

Develop a sense of self
that reflects your history,
values, beliefs, and roles
in the larger community.

Listen to and learn from
others' experience and
ideas

Communicate so that
others understand

Reflect on and
re-evaluate your
opinions and ideas

Form and express
opinions and ideas

Develop a sense of self
that reflects your history,
values, beliefs, and roles
in the larger community.

Listen to and learn from
others' experience and
ideas

Communicate so that
others understand

Reflect on and
re-evaluate your
opinions and ideas

Form and express
opinions and ideas

Strengthen and express
sense of self that reflects
your history, values,
beliefs, and roles in the
larger community.

Learn from others'
experience and ideas,
e. g., listen, read, watch

Communicate so that
others understand

Reflect on and
re-evaluate your
opinions and ideas

INITIAL DRAFT 3/ 97 REVISION 5/ 97 REVISION FINAL
Figure 10. SAMPLE CHANGES IN CITIZEN ROLE MAP

B R O A D A R E A S O F R E S P O N S I B I L I T Y

K E Y A C T I V I T I E S 31.
31 Page 32 33
the structured feedback sessions were also asked to
help identify role indicators. Facilitators introduced
role indicators this way:
"Role indicators help us understand the outcomes,
procedures, and context of what successful adults do as
parents, or workers, or citizensÉ. Role indicators
share several characteristics:
° They are activity-based - they are something some-one
does and usually have an observable outcome or
process.
° They usually require multiple skills and knowledge
to do so: they are not a single skill.
° They usually have an evaluative quality: they sug-gest
how and/ or how well something is done."
[Facil-itators'
guide, undated]
Working in small groups, participants used
their own experience to identify role indicators for
specific subsets of key activities. Each small group
presented its proposed role indicators to the whole

group, where they were further refined (examples
from citizen role in Figure 11).
In all the roles, facili-tators
found that it was often hard for people to
identify role indicators. Role performance seems
grounded in context, and participants found it diffi-cult
to think globally about indicators of perfor-mance.
Most could think of specific examples from
their own experience more easily than general
descriptions. The evaluative aspect was particularly
difficult - describing how well something is done.
In one report from a family role session, the facilita-tor
commented that "12-15 people in a group sel-dom
come up with the evaluative component other
than to add the word 'consistently' or 'regularly. ' Try-ing
to get this from the group has been frustrating."
(Internal e-mail, 5/ 8/ 97).
Because the role indicator data was collected
in the later rounds of feedback sessions, there was
no opportunity to test or validate these in subse-2

8

Equipped for the Future Research Report
Become and stay
informed

Identify and monitor
problems, community
needs, strengths, and
resources

° Asks the right
questions to get
relevant information
° Contacts people
knowledgeable about
the problem or need
° Routinely monitors a
variety of media
resources

Form and express
opinions and ideas

Strengthen and express
sense of self

° Feel comfortable in
diverse situations and
with diverse groups
° Identifies own
perspectives, points
of view, values,
and beliefs
° Exhibits self-confidence
and
personal authority

Work Together
Get involved in the
community and get
others involved

° Supports the efforts of
others
° Reaches out to a
diversity of people
° Volunteers time and
effort

Take action to
strengthen
communities

Help self and others.

° Shares personal
resources (time,
money, materials)
° Assesses personal
needs and strengths
to determine and/ or
inform
° Stays up-to-date on
community resources
and needs.

Figure 11. SAMPLE ROLE INDICATORS: CITIZEN ROLE
B R O A D A R E A S O F R E S P O N S I B I L I T Y

K E Y A C T I V I Ty

R O L E I N D I C AT O R S

In all the roles, facilitators found that it was often hard for people
to identify role indicators. Role performance seems grounded in context, and participants
found it difficult to think globally about indicators of performance.
32.
32 Page 33 34
quent sessions. However, issues relating to role indi-cators
suggested in earlier sessions were passed on to
facilitators at later ones, in an effort to make the role
indicators clear and accurate. Use of the role indica-tor
data in the beginning development of standards
is discussed in Section 6.
The role maps represent the first step toward
developing standards for adult education. They are
not the first attempt to identify skills needed to per-form
life tasks, since they are similar to earlier com-petency-
based efforts. However, the role maps differ
in their focus on purpose, meaning, and value.

Coordinators were constantly aware of the hazards
of developing long lists of decontextualised compe-tencies.
Instead they sought to link tasks in a hierar-chy
focused on broad areas of responsibility. On
their own, the role maps provide only part of the
guidance needed by the field. Adult education for
the most part teaches skills and knowledge, which
now needed to be elaborated and linked with the
role maps to show how application and use of skills
relate to real life role performance. The work on
skills and knowledge was the next phase, overlap-ping
with the final refinement of the role maps.

2 9

Equipped for the Future Research Report
The role maps represent the first step
toward developing standards for adult education.
33.
33 Page 34 35
HE ROLE MAPS CONCENTRATED ON BROAD areas
of responsibility, key activities, and role indi-cators
as the areas on which the least work
had already been done. Skills and knowl-edge
were always a crucial element, and new data
had also been gathered on these. In addition, there
was a substantial body of published work on skills
and knowledge associated with adult roles. In the
next phase of work the EFF initiative constructed a
database on skills and knowledge from published
sources and EFF data, and linked the resulting skills
and knowledge data with the refined role maps.

Developing a Database
On Skills and Knowledge
In spring 1997, the EFF technical assistance team
and coding teams from each of the role consortia
reviewed and coded data on skills and knowledge.
Each role consortium was asked to review the liter-ature
relating to its role and identify up to 10 docu-ments
that in its opinion defined the current state
of knowledge about skill requirements for effective
performance of that role (see Appendix C). To com-pile
the sources, the three role consortia consulted
with their national advisory groups, which brought
together representatives of key organizations in
their field.
These documentary sources were uneven in
terms of quality and comprehensiveness. The review
exercise revealed the extent to which there is a need

for additional research on the skills required for
effective role performance, especially for the family
and citizen roles. While the worker role had exten-sive
source documents because of the work of
SCANS, O* NET, and the occupational skills stan-dards,
the citizen and family member roles had far
fewer solid documentary sources. In the area of par-enting,
for example, it became apparent how little of
the large literature on preferred or advised behavior
is based on solid research on the skills and knowl-edge
needed. In the citizen role, although there were
K-12 standards, these reflected a traditional concept
of "civic education" that fails to include the broad
domain of practical "citizenship" identified in EFF's
own research. Both these roles needed to rely heavily
on data collected in EFF focus groups, structured
feedback, and inquiry projects.

Coding the Skills Data
Each of the source documents in the database had
been created independently, and there were few
commonalities in language. The PCI technical assis-tance
team developed an initial coding guide that
made it possible to bring skills and knowledge from
each of the documentary sources into a common
framework, which could then be linked with the role
maps (see Figure 12). Deliberately based on the
Department of Labor's O* NET skills framework, the
guide created a sorting mechanism that could clearly
link the EFF framework with SCANS and O* NET.

3 0

Equipped for the Future Research Report
Identifying Skills and Knowledge

T 34.
34 Page 35 36
There were some significant differences
between this coding guide and the SCANS and
O* NET frameworks.
1. The term "foundation skills" was used in place of
"basic skills." The aim was to broaden the con-ception
of basic skills - the purview of adult
basic education - beyond reading, writing, and
math to address all the EFF skill areas.
2. Technology and learning to learn were included
in Foundation Skills, in accordance with the liter-ature
on occupational skill requirements, which
report these as essential to success. These skills
also appear in SCANS and O* NET, but in other
categories.
3. A new category of "Extended Literacy Skills" was
added to designate skills that are regarded as
essential but go beyond the scope of most basic
education programs. Most of these had been
identified as SCANS "competencies."
Each role consortium identified two coders
experienced in adult education and familiar with
the role maps. The coders for all three role groups
were trained together, and inter-coder reliability was
tested during the training. The technical assistance

team reviewed the coders' work at regular intervals
throughout the coding process and provided assis-tance
to resolve coding problems. The coders
reviewed each source document and highlighted
parts of the text that described skills or knowledge
(called "data text items"). They assigned up to three
codes to a single text item. When a text item did not
fit any of the codes, coders noted issues and prob-lems.
Each text item and code( s) was then entered
into a standard EXCEL spreadsheet. Thousands of
text items were entered, representing the "state of
the art" on adult skill requirements.
The initial coding guide enabled the coders to
create broad categories of skills. To acknowledge
more discrete subskills that reflected the particular
demands of the roles, the coders needed to create
subcategories within each skill category. The coding
frame was revised for each role separately in a series
of "linkage meetings," bringing together the role
coordinator, coders, technical assistance team, and
adult education practitioners with experience in
particular skill areas. Text items within a particular
subcategory of the coding guide, such as reading,
were sorted and re-classified into new "sub-subcate-3

1

Equipped for the Future Research Report
Foundation Skills ° Reading
° Writing/ Drawing
° Mathematics
° Science
° Speaking
° Listening
° Use of Technology
° Learning to Learn

Extended Literacy ° Identifying, Defining,
Skills and Processing
Information
° Resource Management,
Planning, and Scheduling
° Problem Solving,
Decision Making, and
Critical Thinking
° Creative Thinking
° Systems Thinking and
Visioning

Interpersonal Skills ° Guiding and Teaching
Others
° Influencing and
Advocating
° Leading
° Negotiating
° Collaborating
° Valuing Diversity

Personal Development ° Characteristics
° Attributes
° Personal Qualities
° Values and Abilities
° One's View of Oneself

Knowledge ° Academic Knowledge
° Contextual or
Experiential Knowledge

Figure 12. INITIAL CODING GUIDE FOR SKILLS AND KNOWLEDGE 35.
35 Page 36 37
gories." As a result, the original coding guide was
modified differently for each of the three roles: the
first and second levels were the same, but the third
level (sub-subcategories of skills) was different for
each role (see examples of these in Figure 13).

Linking Skills With Role Map Activities
At this point, EFF had two independently derived
documents about each role: a role map consisting
of responsibilities, activities, and role indicators, cre-ated
and refined through original research and
extensive consultation; and a large database of skills
and knowledge, coded mainly from documentary
sources along with original data collected by EFF.
The next step was to bring these two together in
order to achieve the EFF goal of a "clear picture of

what adults need to know and do in order to fulfill
their roles as citizens, workers and family members,"
on which standards could be based.
This task was carried out at the same "linkage
meetings" described above in Coding the Skills
Data. The teams included many of the facilitators of
the structured feedback sessions, who were very well
informed about role map discussions. The teams
were thoroughly familiar with the revised coding
frames. Also on the teams were experienced literacy
practitioners with expertise in particular skill areas.
Working together, participants in all three
linkage meetings created the links between the key
activities of the refined role maps and the skills and
knowledge in the revised coding frame (at the sub-subcategory
level). They were guided in making the

3 2

Equipped for the Future Research Report
CITIZEN ROLE
Apply effective reading
strategies

Comprehend what you read
Read a variety of texts for a
variety of purposes

Use texts to inform opinions
and broaden knowledge
base

Interpret and infer meaning
from text

FAMILY MEMBER ROLE
Read to children
Structure of language
Reading for understanding
Mastering reading

WORKER ROLE
Read or interpret charts, graphs or other
visual displays

Evaluate reading materials
Check against written specifications
Identify personally relevant information from
written documents

Interpret and infer meaning from text
Read to determine actions
Read material and describe concepts
Read numbers
Read and interpret mathematical ideas
Understand vocabulary
Read medical and dental forms and related
information

Read instructions
Read agendas
Read sentences and paragraphs
Apply information gained through reading to
new situations

Figure 13. EXAMPLES OF ROLE-SPECIFIC CODING FRAMES Đ "FOUNDATION SKILLS: READ"

Working together, participants in the three linkage meetings
created the links between the key activities of the refined role maps and the skills
and knowledge in the revised coding frame.
36.
36 Page 37 38
linkage between activities and skills by data collect-ed
earlier in the EFF research. They drew on reports
of feedback session discussions about what the role
activities meant and entailed, as well as the role indi-cators
created in those sessions (see page 27), which
provided a more detailed picture of specific attrib-utes
of key activities.
These linkages were a way of grounding the
skills in their application in key activities, and
ensuring that the key activities are sufficiently elabo-rated
so that education programs can prepare adults
to perform their roles successfully. The goal is to
support adults in their role performance, and for
this, both skills and their practical application are
necessary.
The linkage process reveals how complex the
relationships are between skills and activities. Most
of the activities require several skills. Most of the
skills could be applied in a number of activities. The
"cross-walk" between the two is not a simplistic
equation, but a complex web. It reveals how embed-ded
narrowly-defined "literacy" skills are within
broader interpersonal and intrapersonal skills (like
communication and social skills, self-knowledge and
self-worth) and contextual knowledge. Adults need
skills from all the categories to achieve the purposes
for learning and to carry out their roles effectively.
By the end of June 1997, the EFF team had
accomplished a great deal. They had refined and val-idated
role maps describing broad areas of responsi-bility
and key activities. For each role they had
developed a set of skills and knowledge based on the
literature. They had created linkages between the
role activities and the skills and knowledge needed
to carry them out.
As a basis for standards, the separate roles still
needed to be linked and brought into one coherent
framework. Developing one framework that crosses
the three roles is consistent with life experience.

There may be three roles, but one individual carries
out all three, and that individual does not keep his
or her life in separate compartments. Although the
roles are distinct in many ways, there are many
interconnections and areas of overlap. There is a
great deal of evidence of transfer and interconnec-tions
between learning in one role and performance
in another.
The three roles also needed to be linked from
the perspective of the adult education system. One
set of standards rather than separate role-based
standards would serve the field better. While some
programs focus particularly on work-related learn-ing
or parenting, most support individual learning
across participants' lives. Policymakers and teachers
alike need the clarity and simplicity of a single set of
standards.
For all these reasons, there was a need to con-dense
and abstract the three role maps into one,
without losing the capacity to draw on the finer
detail of the individual role maps. In the next phase
of work, EFF defined "common activities" across
the three roles, and a single set of "generative skills"
that are needed across the roles. Initial work on this
consolidation was carried out by the entire 30-per-son
EFF team at a week-long meeting at the end of
June 1997.

Common Activities
Common activities were defined as those that occur
in all three roles. Because the three role maps had
been constructed separately, common activities were
not necessarily found in the same levels. Sometimes
a "broad area of responsibility" in one role was sub-stantially
the same as a "key activity" in another.
Sometimes a "key activity" in one role occurred as a
"role indicator" in another, reflecting a lesser signifi-cance
to that role. Some of these differences may
reflect the process through which they were identi-3

3

Equipped for the Future Research Report
The goal is to support adults
in their role performance, and for this, both skills
and their practical application are necessary.
37.
37 Page 38 39
fied, as well as the different histories and personal
interpretations of the role teams. But they also
reflect real differences in the centrality of activities,
their importance or frequency in different roles.
In identifying a core set of common activities
across all the roles, all three levels - responsibilities,
key activities, and role indicators - needed to be
compared for all three roles. The entire EFF team
participated in a simple card sort technique to sort
all the role map elements into initial sets of com-mon
items representing similar activities or func-tions.
Four groups worked separately and compared
results. A taskforce took the four proposed sets and

consolidated them into one, which the whole group
reviewed. The goal was to have a set of common
activities whose content did not overlap. That set
was later refined by e-mail to the final set of 12 com-mon
activities (see Figure 14).
Linking activities across the three roles meant
that each role influenced the whole in particular
ways. For example, the citizen and family member
roles were particularly strong on interpersonal and
communication activities such as "Guide and Sup-port
Others" and "Respect Others and Value Diver-sity."
The worker role was particularly strong on
systems activities like "Work Within the Big Picture"

3 4

Equipped for the Future Research Report
Figure 14. COMMON ACTIVITIES
Gather, Analyze, and Use Information
Find and analyze information from diverse sources.
Use it to form opinions, make decisions, and take
action.

Manage Resources
Find, manage, share and allocate time, money, and
material resources. Use resources in a way that
supports your own needs, goals, and priorities and
those of the family, organization, or community.

Work Within the Big Picture
Recognize and monitor the social, economic, politi-cal,
and organizational systems of which you are a
part. Work with their structures, rules, expectations,
practices, and cultures in setting a course of action.

Work Together
Work with family members, neighbors, or coworkers
to get things done.

Provide Leadership
Inspire, influence, direct, and motivate others.
Take responsibility for results.

Guide and Support Others
Help others succeed by setting an example, provid-ing
training, or giving other kinds of assistance.

Seek Guidance and Support From Others
Seek out the support you need from others.

Develop and Express Sense of Self
Examine, clarify, and express your values, beliefs,
culture, and history. Use your understanding of self
to guide your actions.

Respect Others and Value Diversity
Respect and appreciate the values, beliefs, cultures,
and history of others. Use this appreciation to coun-teract
prejudice and stereotypes.

Exercise Rights and Responsibilities
Act and advocate on behalf of yourself and
others based on a knowledge of your rights and
responsibilities and those of others.

Create and Pursue a Vision and Goals
Establish a vision and goals. Use your vision
and goals to identify, plan, and prioritize tasks
and activities.

Keep Pace With Change
Look ahead to challenges and prepare for them by
learning new skills, adapting current skills to new
challenges, and learning from your own and others'
experiences.
(Source: NIFL, 1998)

Linking activities across the three roles meant
that each role influenced the whole in particular ways.
38.
38 Page 39 40
and "Keep Pace With Change." Activities were only
designated as "common" if they appeared in all three
roles. However, common activities had different
emphases and meanings in the context of each role.

Generative Skills
Three separate role-specific skills coding frames had
been created, which nevertheless had a great deal of
commonality and overlap. The consolidation of
these into a common set of "generative skills" was
driven by the same need as that for common activi-ties:
the need of both policymakers and practition-ers
for a single set of standards for adult education.
At the same week-long meeting, EFF team members
sorted and compared the sub-subcategories of each
of the three role-specific skill frameworks, and cre-ated
a first draft of a consolidated list of skills that
occur in all three. This was later refined via e-mail
correspondence and the work of the technical assis-tance
team. At one point the list grew to over 50
skills, clearly falling into the "proliferation" trap that
Sticht had identified a year earlier as problematic for
all competency frameworks. The long list needed to
be consolidated and reduced to something more
manageable. In the end, 17 generative skills in four
broad categories were identified (Figure 15).
Generative skills were defined as "integrated
skill processes that are durable over time, in the face
of changes in technology, work processes, and soci-etal
demands." As such they are required in order to
carry out the common activities identified from the
role maps, and many day-to-day tasks. Using the
original role-specific skills coding, which linked
skills with key activities, the new skill classification
could be directly linked with the newly defined
common activities.
While some of the skills appear familiar to
adult educators, their definitions reveal them as
more clearly linked to purposeful action. "Read with

Understanding," for example, is defined as follows:
To read with understanding adults need to
determine the reading purpose; select reading
strategies appropriate to the purpose; monitor
comprehension and adjust reading strategies;
analyze the information and reflect on its under-lying
meaning; integrate it with prior knowledge
to address reading purpose.
14

3 5

Equipped for the Future Research Report
Figure 15. GENERATIVE SKILLS 12
Communication Skills
These five skills enable adults to communicate
ideas, information, and opinions to diverse
audiences.
° Read With Understanding
° Convey Ideas in Writing
° Speak So Others Can Understand
° Listen Actively
° Observe Critically

Interpersonal Skills
These five skills enable adults to work with others.
° Cooperate With Others
° Advocate and Influence
° Resolve Conflict and Negotiate
° Guide
° Lead 13

Decision-Making Skills
These three skills hold the keys to making
decisions, solving problems, formulating action
plans, and evaluating results.
° Plan
° Solve Problems
° Use Math to Solve Problems and Communicate

Lifelong Learning Skills
These four skills enable adults to keep learning in
order to keep up with change.
° Take Responsibility for Learning
° Reflect and Evaluate
° Learn Through Research
° Use Information and Communications Technology

The consolidation of the role-specific skills coding frames into a common set of
"generative skills" was driven by the same need as that for common activities:
the need of both policymakers and practitioners for a single set of standards for adult education.
39.
39 Page 40 41
"Solve Problems" is defined as follows:
To solve problems adults need to anticipate or
identify problems, use information from diverse
sources to arrive at a clearer understanding of
the problem and its root causes; generate alter-native
solutions, evaluate strengths and weak-nesses
of alternatives; select the alternative that
has the best chance of solving the problem; and
establish criteria for evaluating the effectiveness
of solution.
The generative skills are broad and include
other, more specific skills. "Read with Understand-ing,"
for example, assumes decoding and pre-read-ing
strategies to comprehend and interpret text. But
"Read with Understanding" is more than a simple
additive effect of more specific skills: it requires a
critical, evaluative stance to reading, a view of read-ing
that is conceptual, not mechanical. So it is with
most of the generative skills: they are skills that cen-ter
on humans as purposeful, meaning-making
beings. They are consistent with the theoretical
assumptions underlying EFF and with the concepts
of the roles derived from participants in focus
groups and feedback sessions.
The skills link with the key activities, but
there are clearly many ways of getting an activity
accomplished. While reading may be regarded as
an essential skill for gathering and using informa-tion,
there are many who successfully use other
skills in that activity - talking and listening with
others, for example. The skills are like a toolbox, in
which we all are quite proficient at some and less
proficient at others. Our tools get better as we use
them and may atrophy if we don't. We select from
our tool box to meet the needs of particular tasks,
in particular contexts, at particular times. Learning
helps us have more skills from which to select, and
be more proficient with our skills.

Knowledge Domains
Knowledge domains had been coded along with
skills from the documentary sources in the role-specific
databases, with coders for each role elabo-rating
the initial two broad codes for knowledge
(academic and practical/ experiential). These were
then compared and consolidated across the three
roles, in the same way as the activities and skills, to
create a set of "Knowledge Domains" that cross the
three roles. Knowledge domains were defined as
"the concepts, procedures, data, information, and
perspectives that support the generative skills and
are necessary to carry out the common activities in
our adult roles."
It was clear from the coding process that
knowledge domains are more context-specific than
either skills or activities. The particular knowledge
needed in a certain workplace is different from that
needed in others. The skills and knowledge needed
by parents of a certain age child differ from those
needed by parents of older or younger children. Yet
the whole of our education system is predicated on
the presence of a common knowledge base that
crosses life circumstances and is needed by the "edu-cated
person." The controversies around K-12 stan-dards
have concerned just what should be in that
knowledge base - the relative importance of the
Mogul empire versus the French Revolution, the
importance of trigonometry versus statistics,
whether literature should include works from out-side
the English-speaking world. EFF's focus is on
adult life and the knowledge needed to accomplish
all the adult roles successfully. The knowledge
domains are conceptual rather than detailed; the
specifics emerge in relation to the activities them-selves.

Each piece of the puzzle needed to create stan-dards
was now in place. Starting outside education

3 6

Equipped for the Future Research Report
The generative skills are consistent with the theoretical assumptions
underlying EFF and with the concepts of the roles derived from participants
in focus groups and feedback sessions.
40.
40 Page 41 42
and working inward, EFF participants created role
maps based on what adults do in their roles as
worker, citizen, and parent. They identified the
broad areas of responsibility adults hold, and the
key, central activities adults pursue. They defined
the broad skills and knowledge needed to carry out
the activities and fulfill the role expectations. All fit
within the frame of the four purposes for learning
that adult learners had elaborated in 1994. The
complete framework was the "big picture," within
which education and training programs could sup-port
learning, and which all adults pursued every
day in their own contexts.

This research report stops here, with the role
maps complete and the standards in the process of
being created. The next research report will deal
with subsequent phases of work:
° Validation of Skills - expert validation (by role
experts) and field validation (by practitioners in
field development phase);
° Standards Development and Testing - the field
review phase;
° Performance Measurement - development of new
tools;
° Staff Development and Technical Assistance to
Enable Programs to Implement EFF.

3 7

Equipped for the Future Research Report
How We Grow and Develop: includes knowledge
about physical and intellectual growth as well as
spiritual and psychological development.

How Groups and Teams Work: includes knowledge
about the purposes of groups and teams, the stages
of their development and dynamics, and the
processes that make groups and teams effective.

How Systems Work: includes understanding the
nature and structure of formal and informal systems,
how power is distributed in systems, the official and
unofficial rules operating within a system, and how
various systems interact, such as family, schools,
health care, and social services.

Rights and Responsibilities: focuses on under-standing
the fundamental concepts that are central
to democratic ways of life, including the provisions of

the Constitution of the United States and the rights
and responsibilities of citizenship; political and legal
processes and rights, consumer rights, landlord and
tenant rights and responsibilities; and employment
agreements and union contracts.

Culture, Values, and Ethics: includes knowing the
meaning of traditions and culture in our lives, the
influence of language on culture, how individuals are
shaped by family and community values, and the
effects of values and ethics on law and government.

How the Past Shapes the World We Live In:
includes understanding the historical context of
current issues and opportunities, knowing more
about what came before, and lessons learned.
The historical contexts include family, community,
workplace, nation, and world.

Figure 16. KNOWLEDGE DOMAINS
The knowledge domains are conceptual
rather than detailed; the specifics emerge in relation
to the activities themselves.
41.
41 Page 42 43
QUIPPED FOR THE FUTURE AS A SYSTEM
reform initiative is far from complete,
and it is too early for a thorough
assessment of the work. But over this
five-year period, a great deal has been accomplished
to change the frame of reference of the field of adult
basic education. In this final discussion section we
will review briefly two main areas in which EFF has
important implications: its contributions to inte-grating
theory and practice, and its commitment to
a participatory process for system reform.

Integrating Theory and Practice
EFF builds on and contributes to the growing body
of approaches to learning as a purposeful act - not
decontextualized and value-free, but embedded in
particular purposes and specific contexts. This is
most clearly seen in the complex web linking skills
and activities (the application of skills). There are
no simple or one-to-one relationships; instead,
individuals make choices about how to apply the
skills they have. Essentially, like making dresses out
of flour sacks, adults select skills according to what
is needed in particular applications and what skills
are available to them. Learning increases skills' pro-ficiency
and enriches understanding, providing
adults with greater flexibility in achieving their
purposes.
EFF provides evidence for this theoretical

stance from the systematic input of learners and
stakeholders. At the same time it provides some
insights into the practical application of these differ-ent
concepts of learning in both accountability and
teaching systems.
EFF's focus on the application, rather than
possession, of skills and knowledge is an important
contrast to the educational standards movement,
whose prime focus is possession of a body of knowl-edge.
The requirements of working with the field
demand attention to both application and skills.
The EFF project represents the first time there
has been a concerted attempt to map the major
adult roles. Although a great deal of work had been
done around the worker role, much less had been
done on the citizen and parent/ family member
roles. In these two roles in particular, EFF has made
a useful contribution to specialist fields. For exam-ple,
study of the "civil society" has been growing in
recent years, and EFF's work on the citizen role adds
greatly to our understanding of citizenship (for
more discussion, see Merrifield, 1997).

Learning From the Process
The process of EFF has been one of the most extensive
consultation and participation processes ever con-ducted
in adult education. The vision is grand, but
feet have been planted firmly on the ground through
the ongoing involvement of learners and teachers.

3 8

Equipped for the Future Research Report
Discussion

E 42.
42 Page 43 44
EFF staff now know a lot more about how to do this
kind of project than they did at the beginning.
As the process has developed, some built-in
tensions have become apparent. These are what
Senge calls "creative tensions," which are not neatly
resolvable but without which the project is neither
interesting nor important. They are both EFF's
strength and its greatest challenge.
One such tension is between creating the "big
picture," a common framework within which every-one
can find a space, and honoring and paying atten-tion
to the specific social context in which each
individual learner lives and the particular purposes
for each's learning. The more distilled the framework
gets and the further from the role maps, the harder it
is to stay in that social and purposeful context. Yet
that context is where adults need to act, and it is
what learning needs to address. EFF is seeking a dif-ferent
learning guide from the "skills in isolation"
approach. The tension is the essence of the approach,
but nonetheless hard to manage.
The tension between big picture and particular
learning needs is paralleled by the tension between
creating an accountability structure and supporting
effective instruction. Effective teaching focuses on the
small print of learners' interests and contexts - on
what Sticht calls small growth steps. Accountability
focuses on broad commonalities, common goals, and
comparable achievements. The demands for detail
are very different. Accountability structures require
standards and performance indicators. Teaching
requires processes and content. In trying to work on
both together and create links across them, EFF's task
is infinitely more challenging. But in the common

ground lies the potential for systemic change.
Throughout these last five years, EFF staff have
tried to keep a holistic view of where the project is
going while at the same time working intensely on
small pieces of the framework. Responsiveness to
constituents who have some stake in the outcome is
essential, but means the whole is always being mod-ified
by the development of the parts. This is an iter-ative
approach to theory building that has been
widely accepted in social science (see discussion on
pages 7 and 8 of naturalistic inquiry, grounded theory,
action research, and inquiry).
EFF adopted this
approach in order to generate system reform that
would be credible and acceptable to the field of
practice, to stakeholders, and to policymakers.
The journey mapped in this report is only the
beginning. Since 1997, EFF partners have drafted
and field tested content standards, and are now
working on identifying performance levels. This
work will be reported in the next research report.
Fifteen states are now actively pursuing EFF as an
accountability framework that will help them link
program improvement and learner progress.
EFF's most significant accomplishment has
been to shift thinking about the purpose of adult
education. From a conception that adult education's
work is to replicate K-12 education and teach
knowledge and skills that are autonomous and inde-pendent,
EFF has pushed us toward thinking of
adult education as preparing people for the future
by teaching skills and knowledge needed to be more
effective workers, parents, and citizens. Equipped for
the Future is a unique and remarkable effort to
mold adult education for the next century.

3 9

Equipped for the Future Research Report
Throughout these last five years, EFF staff have tried to keep
a holistic view of where the project is going while at the same time
working intensely on small pieces of the framework.
43.
43 Page 44 45
1 In 1993 when this project started, the present
Goal 6 was actually Goal 5, and students were
invited to comment on Goal 5. In 1994 Congress
added two more goals to the original list, which
resulted in renumbering. The wording was not
changed. To avoid confusion we will refer here
only to Goal 6.

2 The Association for Community Based Educa-tion,
Laubach Literacy Action, Literacy Volunteers
of America, the National Association of State
Directors of Adult Education, the Student Coali-tion
for Adult Literacy Education, and United
Way of America's Literacy Initiative.

3 Only LLA provided its database to NIFL; the
other organizations undertook to distribute
themselves. We do not know if each of the 6,000
reached a discrete program - there is likely to be
duplication among the various mailing lists. If
6,000 reached students, with 1,500 responses the
response rate would be 25 percent. If the 6,000
mailings reached programs, with 149 programs
responding the response rate would have been
2.5%. Given possible duplication between lists,
and some distributors who did not follow
through, the real response rate is likely to have
been much higher.

4 Technical support for the analysis phase was pro-vided
by the Graduate School of Education and
Human Resources at the George Washington
University. Dr. Ray Rist, Director of the Center for
Policy Studies, an experienced qualitative

researcher, headed the research team, along with
Dr. Gregg Jackson, who worked with the coding
team in identifying and refining themes, and Dr.
David Wizer, who oversaw technical aspects of
using Ethnograph software to manage the data
analysis. Four GWU graduate students were
coders, and Dr. Sondra Stein from NIFL complet-ed
the team.

5 This team read approximately 80% of the initial
data. Files were exchanged so that they were read
by more than one team member. Discussions of
possible themes took place at staged intervals.

6 The coding team was the four GWU graduate
students, under the supervision of Dr. Jackson,
and working closely with Dr. Stein.

7 In the guide used by coders, each sub-category is
illustrated by sample statements that exemplify it
and provide guides on what should be included.

8 Goal 6 reflects two of these (citizen and worker),
and policy interest in the third - parent/ family
member - is demonstrated in the Even Start legis-lation
addressing family literacy, the America
Reads federal initiative and revisions of the Adult
Education Act (the 1998 Workforce Development
Act). The adult learner consultation made clear
the importance of the parent/ family member role
in people's lives and purposes for learning - in
fact, children and families were the center of their
lives. They wanted better jobs to give their families
a better life. They wanted better communities to

4 0

Equipped for the Future Research Report
Footnotes 44.
44 Page 45 46
have a better place for their children to grow up.
This role could not be left out of adult education.

9 Citizen role: synthesized by the Center for Litera-cy
Studies, drawing on data from Southern
Appalachian and New England Civic Participa-tion
Project; Philadelphia Mayor's Commission
on Literacy - Adults as Citizens; Minneapolis
Public Schools; CONSABE, San Diego, CA; Maine
Statewide Standards Development; and Adult
Numeracy Practitioners' Network.
Worker role: synthesized by Performance Con-sulting
Inc., drawing on data from SCANS,
O* NET, NJAS, North Carolina Literacy Resource
Center; Minneapolis Public Schools; CONSABE,
San Diego, CA; Maine Statewide Standards Devel-opment;
and Adult Numeracy Practitioners' Net-work.
Reviewed for consistency with O* NET by
Michael Campion, Graduate School of Manage-ment,
Purdue University.
Parent/ Family Member role: synthesized by
National Center for Family Literacy, drawing on
data from NCFL project; Minneapolis Public
Schools; CONSABE, San Diego, CA; Maine
Statewide Standards Development; and Adult
Numeracy Practitioners' Network.

10 Characteristics may add to more than total when
individuals declared more than one affiliation,
and may add to less than total when participants
did not declare demographic data..

11 Some programs offer both ABE and ESOL, so
total adds to more than the number of programs.

12 See Appendix D for May, 1999 definitions of each
of these generative skills.

13 In the standards review process it was determined
that "lead" did not hold up as a separate and
assessable skill. Participants in the field and expert
review process recommended that we maintain
the Common Activity "Provide Leadership, " sup-ported
by a range of interpersonal, decision-mak-ing,
and lifelong learning skills.

14 Revised April, 1999, as a result of the standards
review process.

4 1

Equipped for the Future Research Report 45.
45 Page 46 47
Behn, Robert D. (1993), "The Growing Use of Perfor-mance
Indicators in State Government," in Robert D.
Behn et al, Performance Indicators for State Government:
A Report to the Nation's Governors and Their Chiefs of
Staff,
Durham, NC: The Governors Center at Duke
University.

Brizius, Jack A. and Campbell, Michael D. (1991), Get-ting
Results: A Guide for Government Accountability.
Washington DC: Council of Governors' Policy Advisors.

CONSABE (1996), Voices from the Community. Working
Draft Report on Part 1: Working Toward Draft Standards.
Submitted to NIFL July. San Diego: CWELL (Consortium
for Workforce Education and Lifelong Learning).

Deming, W. E. (1986), Out of the Crisis. Boston MA: MIT
Press.

Fellegi, Ivan and Alexander, Thomas J. (1995), Literacy,
Economy and Society: Results of the First International
Adult Literacy Survey.
Paris: Organization for Economic
and Community Development (OECD).

Gagnon, Paul (1995), "What Should Children Learn?"
Atlantic Monthly, December, pp. 65-74.

Glaser, Barney G. and Strauss, Anselm L. (1967), The
Discovery of Grounded Theory: Strategies for Qualitative
Research.
London: Weidenfeld and Nicolson.

Goetz, J. P. and LeCompte, M. D. (1984), Ethnography
and Qualitative Design in Educational Research.
Orlando,
FL: Academic Press.

Guba, Egon G. and Lincoln, Yvonna S. (1982), Effective
Evaluation: Improving the Usefulness of Evaluation Results
through Responsive and Naturalistic Approaches.
San
Francisco: Jossey-Bass Publishers.

Kegan, Robert (1982), The Evolving Self. Cambridge,
Mass: Harvard University Press.

Kegan, Robert (1994), In Over Our Heads: The Mental
Demands of Modern Life.
Cambridge, Mass: Harvard
University Press.

Kirsch, Irwin, Jungeblut, Ann, Jenkins, Lynn, and Kol-stad,
Andrew (1993), Adult Literacy in America: A First

Look at the Results of the National Adult Literacy Survey.
Washington DC: U. S. Government Printing Office.

Knowles, Malcolm S. (1980), The Modern Practice of
Adult Education: From Pedagogy to Andragogy.
Engle-wood
Cliffs, NJ: Cambridge Adult Education.

Lave, J. and Wenger, E. (1991), Situated Learning: Legiti-mate
Peripheral Participation.
New York: Cambridge
University Press.

Merrifield, J., Bingman, M. B., Hemphill, D. and deMar-rais,
K. B. (1997), Life at the Margins: Literacy, Language,
and Technology in Everyday Life.
New York: Teachers Col-lege
Press.

Merrifield, J. (1997), "Finding Our Lodestone Again:
Democracy, the Civil Society and Adult Education,"
paper presented to 27th annual SCUTREA international
conference, Crossing Borders, Breaking Boundaries, at
Birkbeck College, University of London, July.

Mezirow, J. and Associates (1990), Fostering Critical
Reflection in Adulthood: A Guide to Transformative and
Emancipatory Learning.
San Francisco: Jossey-Bass.

National Commission on Excellence in Schools (1983),
A Nation at Risk: The Imperative for Educational Reform.
Washington DC: U. S. Government Printing Office.

National Council of Teachers of Mathematics (1989),
Curriculum and Evaluation Standards for School Mathe-matics.
Reston, VA: Author.

National Governors Association (nd), Performance-Based
Governance.
Strategy Group on Performance-Based
Governance, NGA.

National Institute for Literacy (NIFL) (1995a), A Review
of the Oregon and Texas Experience in Building Perfor-mance
Measurement and Reporting Systems,
Prepared by
Michael Campbell, Brizius and Foster. Washington DC:
Author.

National Institute for Literacy (1995b), Literacy Works.
Washington DC: Author.

National Institute for Literacy (NIFL) (1995c), "Adult
Learning System Reform and Improvement Planning

4 2

Equipped for the Future Research Report
References 46.
46 Page 47 48
grant application for planning grant awards to launch a
collaborative, grassroots process of system reform and
improvement for adult literacy and basic skills instruc-tion."
Federal Register, July 12 1995: 35949f.

National Institute for Literacy (NIFL) (1998), Equipped
for the Future: A New Framework for Adult Learning
(poster). Washington DC: Author.

Osborne, David and Gaebler, Ted (1993), Reinventing
Government: How the Entrepreneurial Spirit Is Trans-forming
the Public Sector.
New York: Plume, originally
published Reading, Mass: Addison-Wesley, 1992.

Reason, Peter (1994), "Three Approaches to Participative
Inquiry," in Denzin, N. K. and Lincoln, Y. S. (eds.) Hand-book
of Qualitative Research.
Thousand Oaks, CA: Sage
Publications, pp. 324-339.

Romberg, Thomas A. (1993), "NCTM's Standards: A
Rallying Flag for Mathematics Teachers," Educational
Leadership,
February: pp. 36-41.

Secretary's Commission on Achieving Necessary Skills
(SCANS) (1991), What Work Requires of Schools: A
SCANS Report for America 2000.
Washington DC: US
Department of Labor.

Senge, Peter (1991), The Fifth Discipline: The Art and
Practice of the Learning Organization.
New York: Dou-bleday.

Stagg, Debra D. (1992), Alternative Approaches to Out-comes
Assessment for Postsecondary Vocational Education.
Berkeley, CA: National Center for Research in Vocational
Education.

Stein, Sondra Gayle (1993), Framework for Assessing Pro-gram
Quality,
Washington DC: Association for Commu-nity
Based Education (ACBE).

Stein, Sondra Gayle (1995), Equipped for the Future: A
Customer-Driven Vision for Adult Literacy and Lifelong
Learning.
Washington DC: National Institute for Literacy.

Stein, Sondra Gayle (1997), Equipped for the Future: A
Reform Agenda for Adult Literacy and Lifelong Learning.
Washington DC: National Institute for Literacy.

Swadley and Ziolkowski (1996), National Institute for
Literacy PMRIS Evaluation: Final Report, Cross-site Eval-uation
and Case Studies,
Prepared for NIFL, Washington
DC: National Institute for Literacy.

Weiss, Lori (1995), Adult Education: Measuring Results
Has Been Challenging.
Washington DC: U. S. General
Accounting Office.

Wenger, E. (1996), Communities of Learning. New York:
Cambridge University Press.

Young, M. B., Fleischman, H., Fitzgerald, N. and Morgan,
M. A. (1995), National Evaluation of Adult Education Pro-grams.
Prepared for U. S. Department of Education, by
Development Associates Inc.

4 3

Equipped for the Future Research Report 47.
47 Page 48 49
1. Adult Numeracy Practitioners Network.
Looked across all three adult roles, but focused on
the numeracy aspects of each. The process involved
nearly 300 individuals in seven states. These includ-ed
21 learner focus groups in seven states (six each
in Illinois and Virginia, four in Oregon, two in Ohio,
one each in New Hampshire, Vermont and Rhode
Island). 171 adult learners participated, all enrolled
in adult education mathematics classes.

The learners participating were:
° 59% were female.
° 71% were urban.
° 50% were white; 26% were African American; 12%
Hispanic; 7% Asian and 3% Native American.
° 69% were parents.
° 60% were unemployed.
° Almost half (49% were participating in GED
classes, 26% in adult basic education, and 25% in
ESL, workplace and developmental college classes.
Three focus groups were held in correctional
facilities.

In addition to the adult learner focus groups, five
stakeholder focus groups were held, convening
stockholders from Illinois, Massachusetts, Ohio,
Oregon and Virginia. Most of the 61 stakeholders
were involved in adult education, training or
employment - state and municipal administrators,
college and university personnel, staff developers,
publishers and employers.

Data were also collected from a "virtual" study
group of mathematics teachers, graduate students
and researchers from the US and other countries
who communicated via an electronic discussion
network.

Five teacher study groups involved 41 teachers from
the four states of Illinois Ohio, Oregon and Virginia
as well as the New England Regional Math Group
which included teachers from all six New England
states. These teachers came from a variety of set-tings:
community colleges, correctional facilities,
local education agencies, and community-based
organizations.

A Working Group consisted of representatives from
each of the regions participating in the project,
which already had active math teams connected to
ANPN.

2. National Center for Family Literacy
NCFL partners convened a total of 29 focus groups
involving 223 individuals in five states - Arizona,
Washington, Louisiana, Kentucky and Tennessee.
These included 12 learner focus groups, six stake-holder
groups, three groups which combined stake-holders
and practitioners, seven practitioner groups,
and one mixed focus group.

The learners participating were:
° 94% female.
° 28% white, 30% African American, 24% Hispanic,

4 4

Equipped for the Future Research Report
APPENDIX A.
1995-96 Planning Projects:
Focus Group Participation 48.
48 Page 49 50
4 5
Equipped for the Future Research Report
16% Native American, and 1% Asian/ Pacific
islander.
° 19% were employed.
° 42% were married.
° 69% were native English speakers.
° 82% spoke English primarily at home.

The stakeholders participating in the focus groups
were:
° 85% female.
° 52% white, 16% African American, 5% Hispanic,
8% Native American and 1% Asian/ Pacific
islander.
° They had been involved in adult education from
1 to 39 years, with a mean of 8 years.
° Their age range was from 19 to 69, with a mean
of 43 years.

3. Southern Appalachian and New England
Civic Participation Project.
The two partners in the Civic Participation Pro-ject -
the Center for Literacy Studies at the Universi-ty
of Tennessee and the New England Literacy
Resource Center - convened two Working Groups
(one in each region) and conducted a total of eight
focus groups and eight inquiry projects, involving a
total of 172 people. Focus groups were conducted in
Kentucky, Virginia, Tennessee, Vermont, and Massa-chusetts,
and in addition to participants from these
states involved people from Rhode Island, New
Hampshire, and Maine (a total of eight states).

The focus groups included adult literacy and ESOL
practitioners, policy makers, stakeholders, learners,
and people active in civic life in various ways -
members of grassroots community organizations,
state and local elected officials, volunteers, social
services and health care providers. They were:
° 57 women and 28 men.
° 69 white and 16 people of color.

The inquiry projects were unique to the Civic Par-ticipation
Project. These were designed to explore
the meaning of civic participation in the lives of

adult learners and their teachers, and to begin to
document the ways in which the topic is incorporat-ed
into ABE and ESOL classrooms. They involved a
teacher and class of students at each of 8 adult edu-cation
programs in Tennessee, Virginia, Rhode
Island and Massachusetts. In each project the teach-ers
and students explored the meaning of civic par-ticipation
over 10-12 class sessions, and produced
documentation in terms of discussion notes, stu-dent
writings, drawings and teacher journals. The
Appalachian inquiry projects also developed an
action step, in which the teacher and students iden-tified
an issue about which they were concerned and
took some action. Close to the end of the projects a
CPP staff member conducted a group interview
with each class about what they had learned, and to
capture their reflections on the skills and knowledge
they had needed to take action.

4. Maine State-Wide Standards Development
Maine looked across all three adult roles, and
worked across the state. Maine's work to develop
state-wide standards for adult basic education had
begun before the Equipped for the Future project,
and linked with it through adoption of the four pur-poses
from the 1995 report as the guiding frame-work.
As part of the planning project, Maine
convened five focus groups with a total of 60 partic-ipants.
These included one stakeholder focus group,
one adult education director group, one adult edu-cation
teacher group, and three learner groups. In
addition, Maine used data from a 1995 Summer
Institute of adult education practitioners (input
from 185 people), and convened an Interagency
Working Group of representatives from state agen-cies
and organizations involved in adult education,
employment and training, economic and commu-nity
development, and business and political life.

5. CONSABE: Content Standards for Adult
Basic Education (San Diego, California)
The CONSABE project was developed by CWELL,
the San Diego Consortium for Workforce Education
and Lifelong Learning, based at San Diego Commu- 49.
49 Page 50 51
nity College. Seven focus groups were convened in
San Diego to look across all three adult roles. These
involved a total of 69 people: these included three
groups of ABE students, one group of ABE teachers,
two involving stakeholders from the business, gov-ernment
and education communities, and one of
graduate students at San Diego State University.
Participants were:
° 35 adult learners, 25 stakeholders, 9 constituents.
° 31 males and 37 females.
° age range from 16-21 up to 45+, with most in the
31-45 age group.
° 31 white, 17 Hispanic, 14 African American, 14
Asian/ Pacific Islander and 1 Native American/
Alaskan.

6. Philadelphia Mayor's Commission
on Literacy: Adults as Citizens
MCOL focused on the citizen role, and worked
within the city of Philadelphia. They convened a
Working Group of stakeholders from diverse con-stituencies -
business, adult education and other
educational agencies, government agencies, orga-nized
labor, organized religion, funders and social
service agencies. Three in-class discussion/ research
sessions were held with learners to explore meaning
and actions around citizenship.

A total of seven focus groups were conducted,
involving 29 individuals. These were composed of
individuals from the following stakeholder sectors:
learners, educators, business and labor leaders, poli-cy
makers, funders, community leaders and clergy,
government and media. The first five focus groups
discussed personal citizenship experiences and what
knowledge and skills were needed for them to be
successful, and also reviewed and gave feedback on
draft written standards which had been prepared
using existing published literature. Two subsequent
focus groups were conducted to enhance input on
citizenship without providing input on the draft
standards. Finally, an expert review of the revised
draft standards was conducted by a panel of profes-sionals
in education, civic education and policy.

7. Minneapolis Public Schools
Coordinated by the Adult Literacy Program of the
Minneapolis Public Schools, this project looked
across all three adult roles within the city of Min-neapolis.
A Working Group was convened with rep-resentatives
from adult education and training,
business and labor, community organizations, and
social services. Twenty focus groups were convened:
12 groups of adult learners (106 individuals). These
were:
° 38% African American, 26% Southeast Asian, 11%
white, 10% Native American, 7% African, and 5%
Hispanic.
° ranging in age from teens to 60s, with two thirds in
teens and 20s.
° 66% female.

In addition, eight stakeholder groups were held (76
people) - two each of employment counselors and
teachers, employers, community groups, and
ABE/ ESL teachers. These were:
° 78% white, 10% African American, 10% Hispanic,
1% Southeast Asian and 1% African.

8. North Carolina Literacy Resource
Center: Worker Role
The North Carolina Literacy Resource Center
engaged a 22 member Work Group composed of
basic skills/ literacy providers and program adminis-trators,
policy makers and one learner to oversee the
project. It was an active group that met six times
during the year (once in a two-day retreat), was
involved in the design and conduct of data collec-tion
activities and the analysis of data.

The project invited participation of the basic
skills/ literacy community, employers and the gener-al
public in a series of focus groups designed to
explore responsibilities of adults as workers and to
examine skills and knowledge needed by adults in
their worker role.

Seven day-long focus groups were held around the
state. Work Group members were involved in set-4

6

Equipped for the Future Research Report 50.
50 Page 51 52
ting these up, recruiting participants, and facilitat-ing
activities. In addition, Work Group members
hosted five further events: two focus groups with
Hispanic learners at a community college (one con-ducted
in Spanish); one focus group with learners
and one with employers at Goodwill Industries; a
short focus group with student editors of a Literacy
South publication. A total of 229 people attended
these meetings:

° 89 from community colleges.
° 36 from community based literacy organizations.
° 25 from other agencies (including Dept. of Social
Services, Head Start, Employment Security Com-mission).

° 17 represented employers.
° 62 were learners (mostly in the extra focus groups
at the end).
Demographic data on participants were not reported.

4 7

Equipped for the Future Research Report 51.
51 Page 52 53
4 8
Equipped for the Future Research Report
APPENDIX B.
Structured Feedback Process, 1996-97 WORKER ROLE

NO. OF NO. OF ORGANIZATIONAL KEY SERVICES STATE SESSIONS PARTICIPANTS AFFILIATIONS & SECTORS DEMOGRAPHICS
Maine
6 64 Business & industry: 50 Retail food: 29 Male/ female: 23/ 41
Voluntary/ comm. orgs.: 4 Other services: 11 White/ Asian: 62/ 1
Education/ acad.: 13 Education: 9 Aged 18-25: 9
26-35: 11
36-49: 26
50+: 10

North Carolina 5 46 Business & industry: 31 Fabrication/ Male/ female: 21/ 23
Education: 15 machine: 10 White/ African American/
Govt.: 1 Electronics mfg: 7 other: 30/ 12/ 1
Aged 18-25: 2
26-35: 12
36-49: 19
50+: 13

Ohio 5 57 Business & industry: 40 Fabrication/ Male/ female: 19/ 0
Labor: 10 machine: 28 White/ African American/
Education: 21 Hispanic: 47/ 6/ 2
Aged 18-25: 1
26-35: 10
36-49: 31
50+: 13

Vermont 6 129 Business & industry: 40 Health care: 40 Male/ female: 22/ 94
Union: 3 White/ other: 123/ 2
State govt: 2 Aged 18-25: 4
Education/ Acad.: 67 26-35: 25
Community/ voluntary org: 7 36-49: 69
50+: 30

Pennsylvania 1 25 Business & industry: 4 Male/ female: 0/ 25
Union: 25 White/ African American/
Govt: 11 other: 9/ 15/ 1
Education/ Acad.: 1 Aged 18-25: 1
Community/ voluntary org: 10 26-35: 12
36-49: 12
50+: 0

Virginia 5 50 Business & industry: 43 Poultry processing: 11 Male/ female: 16/ 34
Union: 8 Food mfg.: 18 White/ African American:
Food processing: 10 31/ 19
Grocery/ retail: 10 Aged 18-25: 6
26-35: 10
36-49: 27
50+: 7
TOTAL 28 371 52.
52 Page 53 54
4 9
Equipped for the Future Research Report
CITIZEN ROLE
NO. OF NO. OF ORGANIZATIONAL STATE SESSIONS PARTICIPANTS AFFILIATIONS DEMOGRAPHICS*

California 2 30 Interagency Council for Adult Education Male/ Female: 8/ 15
(representatives from corrections, labor, Afro American/ Caucasian/
education, community colleges, libraries). Hispanic/ Other: 2/ 10/ 8/ 3
Aged 18-25: 0, 26-35: 1,
36-49: 13, 50-64: 6, 65+: 0

Connecticut 1 10 Job Center staff, educators, community Male/ Female: 2/ 8
volunteers, Community Development Board, Afro American/ Caucasian/
Urban League. Hispanic: 2/ 5/ 3
Aged 18-25: 0, 26-35: 1,
36-49: 3, 50-64: 5, 65+: 0

Kentucky 1 11 Ag. Extension, Kentuckians for the Male/ Female: 3/ 7
Commonwealth citizen organization, Afro American/ Caucasian: 2/ 9
Kentucky Farm Alliance, Urban Council, Aged 18-25: 0, 26-35: 0
state agencies, adult learning center director. 36-49: 4, 50-64: 6, 65+: 1

Massachusetts 3 18 MA Coalition for Adult Education Male/ Female: 4/ 12
(practitioners), community organizations, Cauc/ Hisp/ Other: 10/ 5/ 1
Latino community organizations. Aged: 18-25: 0, 26-35: 3,
36-49: 8, 50-64: 4, 65+: 0

New Hampshire 1 11 Educators, Latino community organization Male/ Female: 5/ 6
reps., American Friends Service Committee, Afro American/ Caucasian/
community volunteers. Hispanic: 1/ 9/ 1
Aged 18-25: 0, 26-35: 0
36-49: 6, 50-64: 5, 65+: 0

New Mexico 1 21 Staff of migrant education, community Male/ Female: 0/ 2
college, and college preparation programs Hispanic: 2
(national representation). Conducted in Aged 18-25: 0, 26-35: 0
Spanish and English. 36-49: 0, 50-64: 2, 65+: 0

North Carolina 1 10 Staff and learners in community education Male/ Female: 3/ 4
and GED preparation program for migrant Hispanic: 7
workers. Aged 18-25: 2, 26-35: 3
36-49: 1, 50-64: 1, 65+: 0

Pennsylvania 4 34 Businesses, labor, community organizations, Male/ Female: 7/ 23
human services, education, Asian-American Afro American/ Asian/
community organizations, social service Caucasian/ Other: 9/ 3/ 10/ 1
agencies, parent organizations. Aged 18-25: 0, 26-35: 2
36-49: 14, 50-64: 6, 65+: 0

Rhode Island 1 6 Urban League, Progreso Latino, South Male/ Female: 2/ 3
Providence Neighborhood Association, Afro American/ Caucasian/
United Church of Christ, educators. Hispanic: 1/ 3/ 1
Aged 18-25: 0, 26-35: 2
36-49: 2, 50-64: 0, 65+: 1

* Not all participants responded
or replied to all questions

(Continued, next page) 53.
53 Page 54 55
5 0
Equipped for the Future Research Report
CITIZEN ROLE, continued
NO. OF NO. OF ORGANIZATIONAL STATE SESSIONS PARTICIPANTS AFFILIATIONS DEMOGRAPHICS*

Tennessee 4 37 Urban League and other civic organizations, Male/ Female: 8/ 29
elected officials, government officials, Afro American/ Caucasian: 7/ 30
business organizations, religious Aged 18-25: 1, 26-35: 5
organizations, community organizations, 36-49: 16, 50-64: 10, 65+: 4
parent organizations.

Texas 3 38 Workforce Commission, community Male/ Female: 14/ 28
organizations, businesses, Dept. of Afro American/ Asian/
Corrections; Chamber of Commerce, Caucasian/ Hispanic/
religious organizations, migrant and Other: 3/ 3/ 24/ 12/ 1
community education; adult literacy Aged 18-25: 0, 26-35: 4
practitioners and students. 36-49: 24, 50-64: 15, 65+: 0

Vermont 1 9 Educators, school improvement coordinator, Male/ Female: 2/ 5
corrections official, VISTA. Caucasian: 6
Aged 18-25: 1, 26-35: 0
36-49: 5, 50-64: 1, 65+: 0

Virginia 2 23 Community organization members (including Male/ Female: 2/ 21
some ABE students and GED graduates); Afro American/ Caucasian/
Activists in environmental, African American Other: 3/ 18/ 1
and other community organizations, Aged 18-25: 2, 26-35: 3
educators, business leader, staff of state 36-49: 11, 50-64: 6, 65+: 0
elected official, League of Women Voters.

TOTAL 25 257 * Not all participants responded
or replied to all questions

PARENT/ FAMILY MEMBER ROLE
NO. OF NO. OF ORGANIZATIONAL STATE SESSIONS PARTICIPANTS AFFILIATIONS DEMOGRAPHICS*

Arizona 2 29 Family literacy program participants Male/ female: 4/ 25
(Tucson and and other parents. Caucasian/ Am. Indian/
Mesa) Hispanic: 4/ 4/ 21
Aged 18-25: 5, 26-35: 9
36-49: 15, 50+: 0

Wisconsin 3 37 Parents from parent education classes, Male/ female: 5/ 32
(Madison (2) ABE and community college classes. Caucasian/ African American/
and Waukesha) Asian/ Hispanic: 27/ 6/ 2/ 2
Aged 18-25: 3, 26-35: 13
36-49: 14, 50+: 7

Texas 2 26 Parents from the community. Male/ female: 4/ 22
(Houston) Caucasian/ African American/
Asian/ Am. Indian/ Hispanic:
5/ 6/ 1/ 4/ 10
Aged 18-25: 4, 26-35: 7
36-49: 10, 50+: 5

(Continued, next page) 54.
54 Page 55 56
5 1
Equipped for the Future Research Report
PARENT/ FAMILY MEMBER ROLE, continued
NO. OF NO. OF ORGANIZATIONAL STATE SESSIONS PARTICIPANTS AFFILIATIONS DEMOGRAPHICS*

Virginia 2 35 Parents from the community Male/ female: 4/ 31
(Roanoke and Caucasian/ African American/
Richmond) Asian: 22/ 12/ 1
Aged 18-25: 2, 26-35: 7,
36-49: 15, 50+: 11

Louisiana 1 17 Parents from the community Male/ female: 2/ 15
(Bossier City) Caucasian/ African American:
12/ 5
Aged 18-25: 0, 26-35: 8,
36-49: 9, 50+: 0

Kentucky 2 15 Teenage parent program participants Male/ female: 1/ 14
(Louisville) and non-parents Caucasian/ African American:
12/ 3
Aged 18-25: 5, 26-35: 4
36-49: 5, 50+: 1

New Mexico 1 15 Family literacy teachers and parents Male/ female: 3/ 12
(Gallop) from Native American community Caucasian/ Am. Indian/ Hispanic:
4/ 9/ 2
Aged 18-25: 0, 26-35: 2,
36-49: 11, 50+: 2

Pennsylvania 1 21 Parents from the community and others Male/ female: 1/ 20
(Philadelphia) Caucasian/ African American:
4/ 17
Aged 18-25: 5, 26-35: 13,
36-49: 1, 50+: 2

Tennessee 2 41 ABE participants Male/ female: 2/ 39
(Nashville) Caucasian/ African American/
Am. Indian: 27/ 13/ 1
Aged 18-25: 0, 26-35: 10
36-49: 18, 50+: 13

TOTAL 16 236 55.
55 Page 56 57
Citizen Role American Bar Association Special Committee on Youth
Education for Citizenship (1995), Essentials of Law-related
Education: A Guide for Practitioners and Policy-makers.
Chicago: American Bar Association, Division of
Public Education.

Center for Civic Education (1994), National Standards
for Civics and Government,
CA: Author.

Lappe, Frances Moore and DuBois, Paul Martin (1994),
Quickening of America: Rebuilding Our Nation, Rethink-ing
Our Lives.
San Francisco: Jossey Bass.

Massachusetts Department of Education (1995), Habits
of Mind: The Massachusetts Social Studies Framework.
Boston: Author.

Mayor's Commission on Literacy (1996), Content Stan-dards
for Adults as Citizens.
Philadelphia, PA: Author.

National Assessment of Educational Progress (nd.),
Civics Framework for the 1998 National Assessment of
Educational Progress.
Washington DC: National Assess-ment
Governing Board.

EFF data from 1995-96 planning projects.
Reports of teachers in the 1997 inquiry projects.

Parent/ Family Member Role Covey, Stephen (1995), The Seven Habits of Highly Effec-tive
Families.
Covey Leadership Center.
Curran, D. (1983), Traits of a Healthy Family. New York:
Ballantine Books.

Geismar, L. L. et al. (1972), Early Supports for Family
Life: A Social Work Experiment.
PLACE: The Scarecrow
Press.

Geismar, L. L. and Camasso, M. (1993), The Family
Functioning Scale: A Guide to Research and Practice.
New York: Spring Publishing Co.

MotherRead (1989, revised 1995), Child Development
Themes.
Raleigh, NC: Author.

MotherRead (1995), Teacher's Guide: Empowerment
Themes.
Raleigh, NC: Author.

National Center for Family Literacy, (1994) Quality Self
Study (Quality Indicators).
Louisville, KY: Author.

National PTA (1997), National Standards for Parents and
Family Programs.
Washington DC: Author.

Parents as Teachers (1993), Program Planning and
Implementation Guide.
Author.

RMC Research Corporation (1995), Guide to Quality
Even Start Family Literacy Programs.
Washington DC:
U. S. Department of Education.

EFF data from 1995-96 planning projects.

Worker Role American Chemical Society (1994), Foundations for
Excellence in the Chemical Process Industries.
Washing-ton
DC: Author.

American Electronics Association (1994), Setting the
Standard: A Handbook on Skills for the High-Tech Indus-try.
Santa Clara, CA: Author.

Anthony P. Carnevale, Leila Gainer, Ann Meltzer (1990)
Workplace Basics: The Essential Skills Employers Want.
San Francisco: Jossey-Bass Publishers.

Council on Hotel, Restaurant and Institutional Educa-tion
(1995) National Performance Criteria in the Lodging
Industry.
Washington, DC: Author.

5 2

Equipped for the Future Research Report
APPENDIX C.
Documentary Sources for Skills and
Knowledge Database, 1997 56.
56 Page 57 58
Foundation for Industrial Modernization (1994)
National Occupational Skill Standards for Computer
Aided Drafting and Design (CADD).
Washington, DC:
Author.

National Coalition for Advanced Manufacturing (1995),
National Skill Standards Project for Advanced High Per-formance
Manufacturing.
Washington, DC: Author.

National Council of Teachers of Mathematics (1989),
Curriculum and Evaluation Standards for School Mathe-matics.
Reston, VA: NCTM Commission on Standards
for School Mathematics.

National Retail Institute (1994), Raising Retail Stan-dards.
Washington, DC: Author.

National Tooling and Machining Association (1995),
Duties and Standards for Machining Skills: Level I, Level
II.
Ft. Washington, MD: Author.

Secretary's Commission on Achieving Necessary Skills
(1992) What Work Requires of Schools: A SCANS Report
for America 2000.
Washington DC: U. S. Department of
Labor.

U. S. Department of Labor (1995), O* NET Project Sum-mary.
Washington DC: Author.

Elaine Jackson (1994) Non-Language Outcomes in the
Adult Migrant English Program. Sydney,
NSW: National
Centre for English Language Teaching and Research.

EFF data from 1995-96 planning projects.

5 3

Equipped for the Future Research Report 57.
57 Page 58 59
5 4
Equipped for the Future Research Report
APPENDIX D.
Equipped for the Future
Skill Description for Standards

Read With Understanding
° Determine the reading purpose.
° Select reading strategies
appropriate to the purpose.
° Monitor comprehension and
adjust reading strategies.
° Analyze the information and
reflect on its underlying
meaning.
° Integrate it with prior knowledge
to address reading purpose.

Convey Ideas In Writing
° Determine the purpose for
communicating.
° Organize and present
information to serve the
purpose, context, and audience.
° Pay attention to conventions
of English language usage,
including grammar, spelling,
and sentence structure, to
minimize barriers to reader's
comprehension.
° Seek feedback and revise to
enhance the effectiveness of
the communication.

Speak So Others Can
Understand
° Determine the purpose for
communicating.
° Organize and relay information
to effectively serve the purpose,
context, and listener.
° Pay attention to conventions of
oral English communication,
including grammar, word choice,
register, pace, and gesture
in order to minimize barriers to
listener's comprehension.
° Use multiple strategies to
monitor the effectiveness of
the communication.

Listen Actively
° Attend to oral information.
° Clarify purpose for listening
and use listening strategies
appropriate to that purpose.
° Monitor comprehension,
adjusting listening strategies
to overcome barriers
to comprehension.
° Integrate information from
listening with prior knowledge
to address listening purpose.

Observe Critically
° Attend to visual sources of
information, including television
and other media.
° Determine the purpose for
observation and use strategies
appropriate to the purpose.
° Monitor comprehension and
adjust strategies.
° Analyze the accuracy, bias,
and usefulness of the
information.
° Integrate it with prior knowledge
to address viewing purpose.

EFF COMMUNICATION SKILLS 58.
58 Page 59 60
5 5
Equipped for the Future Research Report
EFF DECISION-MAKING SKILLS
Use Math To Solve Problems
And Communicate
° Understand, interpret, and work
with pictures, numbers, and
symbolic information.
° Apply knowledge of
mathematical concepts and
procedures to figure out how
to answer a question, solve a
problem, make a prediction,
or carry out a task that has a
mathematical dimension.
° Define and select data to be
used in solving the problem.
° Determine the degree of
precision required by the
situation.
° Solve problem using appropriate
quantitative procedures and
verify that the results are
reasonable.
° Communicate results using
a variety of mathematical
representations, including
graphs, charts, tables, and
algebraic models.

Solve Problems and Make
Decisions
° Anticipate or identify problems.
° Use information from diverse
sources to arrive at a clearer
understanding of the problem
and its root causes.
° Generate alternative solutions.
° Evaluate strengths and
weaknesses of alternatives,
including potential risks and
benefits and short-and long-term
consequences.
° Select alternative that is most
appropriate to goal, context,
and available resources.
° Establish criteria for evaluating
effectiveness of solution or
decision.

Plan
° Set and prioritize goals.
° Develop an organized approach
of activities and objectives.
° Actively carry out the plan.
° Monitor the plan's progress
while considering any need to
adjust the plan.
° Evaluate its effectiveness in
achieving the goals. 59.
59 Page 60 61
5 6
Equipped for the Future Research Report
Cooperate With Others
° Interact with others in ways
that are friendly, courteous, and
tactful and that demonstrate
respect for others' ideas,
opinions, and contributions.
° Seek input from others in order
to understand their actions and
reactions.
° Offer clear input on own
interests and attitudes so others
can understand one's actions
and reactions.
° Try to adjust one's actions to
take into account the needs of
others and/ or the task to be
accomplished.

Advocate and Influence
° Define what one is trying to
achieve.
° Assess interests, resources,
and the potential for success.
° Gather facts and supporting
information to build a case that
takes into account the interests
and attitudes of others.
° Present a clear case, using a
strategy that takes into account
purpose and audience.
° Revise, as necessary, in
response to feedback.

Resolve Conflict and
Negotiate
° Acknowledge that there is a
conflict.
° Identify areas of agreement and
disagreement.
° Generate options for resolving
conflict that have a "win/ win"
potential.
° Engage parties in trying to reach
agreement on a course of action
that can satisfy the needs and
interests of all.
° Evaluate results of efforts and
revise approach as necessary.

Guide Others
° Assess the needs of others and
one's own ability to assist.
° Use strategies for providing
guidance that take into account
the goals, task, context, and
learning styles of others.
° Arrange opportunities for
learning that build on learner's
strengths.
° Seek feedback on the
usefulness and results of
the assistance.

EFF INTERPERSONAL SKILLS 60.
60 Page 61 62
5 7
Equipped for the Future Research Report
EFF LIFELONG LEARNING SKILLS
Take Responsibility for
Learning
° Establish learning goals that are
based on an understanding of
one's own current and future
learning needs.
° Identify own strengths and
weaknesses as a learner and
seek out opportunities for
learning that help build self-concept
as a learner.
° Become familiar with a range
of learning strategies to acquire
or retain knowledge.
° Identify and use strategies
appropriate to goals, task,
context, and the resources
available for learning.
° Monitor progress toward
goals and modify strategies or
other features of the learning
situation as necessary to
achieve goals.
° Test out new learning in real-life
applications.

Reflect and Evaluate
° Take stock of where one is:
assess what one knows already
and the relevance of that
knowledge.
° Make inferences, predictions,
or judgments based on one's
reflections.

Learn Through Research
° Pose a question to be answered
or make a prediction about
objects or events.
° Use multiple lines of inquiry to
collect information.
° Organize, evaluate, analyze,
and interpret findings.

Use Information and
Communications Technology
° Use computers and other
electronic tools to acquire,
process, and manage
information.
° Use electronic tools to learn
and practice skills.
° Use the Internet to explore
topics, gather information, and
communicate. 61.
61 Page 62 63
The National Institute for Literacy (NIFL) is an independent federal organization that is
leading the national effort toward a fully literate America. By fostering communication,
collaboration, and innovation, NIFL helps build and strengthen national, regional,
and state literacy systems that can better serve adults in the 21st century.

National Institute for Literacy Interagency Group
Secretary of Education, Richard W. Riley
Secretary of Health and Human Services, Donna E. Shalala
Secretary of Labor, Alexis Herman

National Institute for Literacy Advisory Board
Jon Deveaux,
Executive Director, Literacy Partners, Inc.
Mark Emblidge, Executive Director, Virginia Literacy Foundation
Toni Fay, Vice President, Corporate Community Relations, Time Warner
Mary Greene, KERA Public Television, TV Productions
Reynaldo Macias, Professor & Chair, César E. Chávez Instructional Center
for Interdisciplinary Chicana/ o Studies
Marciene Mattleman, Executive Director, Philadelphia Reads
Anthony Sarmiento, Director, Worker Centered Learning, Working for America Institute
Senator Paul Simon, Director, Public Policy Institute
Lynne Waihee, Chair, Read to Me International

National Institute for Literacy Staff
Executive Office
Andy Hartman, Director
Carolyn Y. Staley, Deputy Director
Shelly W. Coles, Executive Assistant

Program Office
Sandra Baxter, Program Director, National Reading Excellence Initiative
Jaleh Behroozi, Program Director, LINCS
Jennifer Cromley, Fellowships Officer
Christy Gullion, Policy Analyst
Susan Green, Program Officer
Wil Hawk, Program Analyst, LINCS
Sara Pendleton, Staff Assistant, Programs
Lynn Reddy, Communications Director
Sondra Stein, Senior Research Associate and Director, Equipped for the Future

Administrative Office
Sharyn Abbott, Executive Officer
Darlene McDonald, Contracts & Grants Assistant
Suzanne Randazzo, Management Operations Specialist

National Institute for Literacy
1775 I Street NW, Suite 730 Washing-ton,
DC 20006-2401

Tel 202-233-2025
Fax 202-233-2050
Web www. nifl. gov National Institute for Literacy NIFL
62.
62 Page 63
National Institute for Literacy
1775 I Street NW, Suite 730
Washington, DC 20006-2401

Tel 202-233-2025
Fax 202-233-2050
Web www. nifl. gov

ED Pubs No. EX 0106

National Institute for Literacy NIFL 63.

Page Navigation Panel

1 2 3 4 5 6 7 8 9
10 11 12 13 14 15 16 17 18 19
20 21 22 23 24 25 26 27 28 29
30 31 32 33 34 35 36 37 38 39
40 41 42 43 44 45 46 47 48 49
50 51 52 53 54 55 56 57 58 59
60 61 62 63