Summary

The Legal Aid Society of Greater Cincinnati has received funding from the 2001 Legal Services Corporation Technology Initiative Grant Program to create a national structure to provide evaluation technical assistance and standards to TIG recipients. The program will be called the Technology Evaluation Project (TEP). Its evaluation system will result in: 

Nowhere in the legal services community is the evaluation of program efforts approached as a core management function in a wide reaching and systematic manner. No standards have been developed for the utilization of technology in a program. TEP envisions working with a national professional expert in evaluation to develop standards. The National Advisory Committee and TEP staff will bring the knowledge of legal services which will ensure that the evaluation is useful and leads to the development of widely accepted standards. Programs will improve their analytic capacity to evaluate their work and their inclination to do so.

The Legal Aid Society of Greater Cincinnati will contract with Management Information Exchange to operate TEP from MIE's Boston office.

Project Narrative

Project Purpose

The Legal Aid Society of Greater Cincinnati has received funding from the 2001 Legal Services Corporation Technology Initiative Grant Program in the national application area to create a national structure to provide evaluation technical assistance and standards to Legal Services Corporation funded programs, in particular Technology Initiative Grant (TIG) recipients. The program will be called the Technology Evaluation Project. The Technology Evaluation Project (TEP) will provide legal services programs with the evaluation standards and evaluation technical assistance they need to fully realize the objectives of their TIG and other technology projects.

Definition of the Problem - Evaluation of a project's success in meetings its goals and objectives is an important part of any project, and yet legal services programs, like many nonprofit organizations, are relatively new to the concept of evaluation, and still struggling with identifica- tion of the elements that make for a useful, and therefore successful, evaluation process. (See Program Evaluation Practice in the Nonprofit Sector, A study funded by The Aspen Institute Nonprofit Sector Research Fund and The Robert Wood Johnson Foundation, Sept. 30, 1998.)

It is particularly important that legal services technology projects funded through the LSC TIG process in 2000 and 2001 be usefully evaluated. LSC is spending $11.25 million over two years to technology initiatives. Technology poses the key to achieving the goal of providing 100% access to legal services and information for the targeted client community. This goal will only be achieved if the learnings and best practices of the technology grant recipients are able to be shared beyond these programs to the legal services community at large. LSC dollars should be leveraged by means of the replication of successful projects throughout the community. Useful evaluation is a particular challenge in this instance because of the many grants in the TIG initiative. 

Creative and Practical Means of Addressing the Problem - The Legal Aid Society of Greater Cincinnati proposes to engage Management Information Exchange (MIE) to develop a national structure to provide evaluation support to Legal Services Corporation funded programs, in particular TIG recipients, for their technology projects. The system will gather data regarding the projects via a common framework to be used as a starting point for evaluation by grant recipients, coupled with support to grantees to answer questions unique to their own projects. Another result of this project will be the development of standards to assist legal services programs in the evaluation of technology projects into the future. As a byproduct, information gathered through this evaluation system will assist LSC in evaluating the success of its efforts through TIG to improve the capacity of legal services programs. 

The TEP evaluation system will have three components. A National Advisory Committee will include the recognized leaders in technology and management in the legal services community, as well as representatives of programs who will be the end users of an evaluation system. This Committee will be responsible for the overall direction of TEP. A TEP Project Manager will have significant management experience and technology technical skills. This Project Manager will be responsible to:

The Evaluation Professional will be identified, through an RFP process, to work as technical assistant to the Project Manager and TEP National Advisory Committee. 

The TEP Project Manager, National Advisory Committee and Evaluation Professional will together review information collected from legal services programs, from technology evaluation resources in the broader nonprofit community, and from their collective experiences, and draft standards for the process of evaluation of technology projects in legal services programs. Such standards will be widely reviewed and commented on by these programs. As they are being developed, the Project Manager, the National Advisory Committee and the Evaluation Professional will immediately begin providing technical support to legal services programs in their evaluation efforts. When the standards have reached a reasonable level of completion, potential evaluators will be recruited from around the community, sent by legal services programs to learn the standards and the process of evaluation, thus making the standards and trained evaluators available to the field by a broadening circle of community members. They will be trained as consultants to provide programs with onsite evaluation capability. 

Anticipated Outcomes and Potential Impacts - TEP's evaluation system will include: 

TEP will result in the creation of an evaluation system for technology projects which will answer the following questions, among others, for grant recipients and LSC:

LSC can anticipate an increase in both the analytic capacity of legal services programs to evaluate other projects and the likelihood that they will do so.

Project Feasibility

Technical Approach - TEP will retain a national expert in evaluation, teaming them with legal services National Advisory Committee members and TEP staff. In this way in the best of both worlds, objective knowledge will be integrated into the legal services setting, making it useful to programs. No decisions have been made as to national expert in evaluation, however one organization has leapt to the fore, being highly regarded in the nonprofit community and providing significant useful work products to that community. This organization, as an example of the resources which exist for this effort, is called InnoNet.

The Qualifications of the Legal Aid Society of Greater Cincinnati - This program has performed consistently well during peer reviews and monitoring visits for the quality of its management systems and legal work. It has been commended for its sound fiscal practices during its audit.

The Qualifications of Management Information Exchange - The Legal Aid Society of Greater Cincinnati will engage Management Information Exchange (MIE) to create TEP. 
MIE's mission is to spark excellence in leadership, management and fundraising in legal services programs serving low income clients, through training, consulting and publishing activities of the highest quality. Since its origin in December 1985, MIE, a nonprofit organization of more than 300 subscriber programs, has been an important management resource for the legal services community. MIE provides legal services programs with training opportunities, fundraising consulting, management consulting, a resource library and the MIE Journal. MIE offers practical information and assistance in managing, supervising, planning, priority setting, working with staff, increasing pro bono efforts, building effective boards of directors, avoiding employment law problems, collective bargaining, using technology, training and developing new resources. MIE is governed by a Board of Directors of legal services managers from around the country. Its work is carried out by members of the Board, active committees of volunteers, and its two full time staff.

MIE has built an excellent reputation for timely and practical training opportunities. To support legal services programs with their leadership and management development plans, it conducts New Executive Director training, Supervising Legal Work, Legal Services Corporation compliance and regulation training, employment law, Managers in the Middle, the National Fundraising Conference, and a changing selection of one-day events on emerging interests including outcome measurement, project management, collaborations and partnerships, program self-assessment and more. Several of MIE trainings are particularly relevant to this Technology Initiative Grant proposal, and demonstrate MIE's capacity to carry out the project for which it will be engaged:

Also relevant to TIG is MIE's experience in delivering fundraising and management consulting services to legal services programs. With an experienced staff member serving as primary expert and consultant, leaders are mobilized from among MIE board and committee members, and the national legal services community to deliver short and long term consulting services to programs. These services are both pro bono and paid.

The management and fundraising consulting activities of MIE are enhanced by its resource library and the MIE Journal. The popular Journal is a one of a kind resource in the legal services community. Published four times each year, the Journal's authors include legal services executive directors, administrators, fundraisers and other staff members, plus consultants, trainers, academicians and other colleagues. Articles are timely and practical, with a focus on the nuts and bolts challenges of managing a legal services program. 

A Special Feature in the Spring 2001 MIE Journal was entitled "21st Century Justice Tools – What Will They Be and How Will We Use Them?" Articles included: 

TEP activities will be carried out over a period of 24 months. The Evaluation Project Manager will be hired during the first two months of the program. The Evaluation Professional will be retained shortly thereafter. During the months of the grant, information will be gathered about TIG projects' evaluation efforts and relationships will be built with the legal services program managers of those projects. The National Advisory Committee, Evaluation Project Manager and Evaluation Professional will develop an appropriate break down of work tasks and calendar. The training of evaluators will take place during year two of the project. Standards will be completed during year two.

Community Involvement

Members of the legal services community play an essential role in the TEP project. National leaders in technology will form the National Advisory Committee which will guide the creation and direction of the project. TIG program recipients will be consulted and provide information to ensure that the project meets their evaluation needs. Leaders in the community will be trained in evaluation through the project. All of these community members will participate in the evaluation of the project on an ongoing basis. In addition, TEP will collaborate fully with all national and multi-state projects funded by TIG to coordinate services and information. The Aspen Study referenced above on nonprofit program evaluation demonstrated convincingly that "participatory" evaluation, where program staff, clients and other stakeholders of a program are engaged in the evaluation process it is far more likely that evaluation findings will be used. 

In the development of this proposal, efforts have been made to engage legal services program leaders in technology from throughout the country as well as national leaders. Consistently they are supportive of creating an opportunity for serious evaluation of technology projects. Some leaders have actively agreed to participate in this proposal by providing us with their credentials, and others have expressed their interest in working with the project in long term.

Evaluation and Documentation

The Legal Aid Society of Greater Cincinnati, though MIE, will evaluate the success of the Technology Evaluation Project. It will gather information from each legal services program user of the TEP; from members of the National Advisory Committee; from volunteer and paid consultants; and from TEP's Project Manager. This evaluation will answer the questions: