Title: 3
Author: Amborski

DEVELOPMENT OF  POLICY CAPACITY IN 

UKRAINIAN  LOCAL GOVERNMENTS

David Amborski					Tom Monastyrski
School of Urban & Regional Planning		People’s Voice Project
Ryerson University					International Center for Policy 
Toronto, Canada					Studies
Amborski@ryerson.ca				Kiev, Ukraine
Tmonastyrski@icps.kiev.ua

Paper Prepared for presentation at the 10^th Annual NISPAcee Conference, Cracow, Poland, April 25-27, 2002
1.0	Introduction

Ukraine emerged from years of communist authoritarian rule in 1991 with an underdeveloped institutional and administrative capacity; the challenges now facing democratic consolidation in the country are abundant. Despite developments such as free elections and freedom of the press, the reality is that the transition to liberal democracy in Ukraine is far from complete. The People’s Voice project (PVP) “Policy Development for Municipal Services Initiative” (hereafter referred to simply as “the Initiative”) is a unique example of public policy building capacity that aims to build open and accountable institutions and facilitate the participation of all Ukrainians in the political, social and economic renewal currently underway in the country.  

This paper will have both an empirical and normative component. The focus of the paper is a pilot program that has the objective of developing policy units and policy capacity in Ukrainian local government. The empirical component will focus on the success of the pilot program that has the objective to develop policy units in two Ukrainian municipalities. The normative component will focus on how municipalities need to focus the budget to ensure implementation of their policies, and furthermore how the budget document and process may be used to aid in the evaluation of policies on an annual basis. 

As indicated, the empirical component will be will be an evaluation of the pilot program being undertaken in two Ukrainian municipalities. This is being undertaken by the People’s Voice Project at the International Center for Policy Studies, (ICPS), in Kyiv. The project is using an approach and content similar to ICPS’s project to develop policy capacity in a number of the Ukraine’s central government ministries. The overall purpose of the project is to provide necessary skills for local government public servants to effectively develop policy and programs that systematically take into account local stakeholders, including citizens. 

The ten month project is being undertaken in two municipalities, Ternopil and Ivano-Frankivsk. Each city has identified a number of policy units that have been given the task of developing policy papers in their respective topic areas. In each city, officials selected staff to participate in each of the six specified policy groups. The people selected generally included both very experienced employees such as department heads and some “rising stars” that are expected to make significant contributions to their municipality in future years. The policy groups include the following topics; education, human resources, budgeting and finance, strategic planning, transportation and housing. Each municipality was required to have policy groups in general administration areas, i.e. strategic planning, human resources, and budgeting/finance. They were then permitted to select three additional policy areas. The project began with seminars on the municipal policy process and priority setting in local government. After the initial seminars, consultations were held with each of the groups to identify the policy issues that they were to address. They were also given initial assignments to develop the problem definition of the issue, and identify the objectives required of a policy. Feedback and consultation was provided on policy topics, such as evaluation and project management, to assist preparation of the final policy papers. The final policy documents were required to be submitted in January (2002). This date proved impossible to meet due to the impending elections held at the end of March in the Ukraine. Due to pressing municipal matters related to the elections, the groups submitted their papers much later than originally expected.

The paper will draw conclusions on the success of the pilot project in terms of building policy capacity in the two cities and make recommendations for how this learning exercise should be delivered in other Ukrainian municipalities. The normative considerations are being used to make recommendations in the context of the pilot study evaluation regarding how the policy documents may be integrated with the budgets process in Ukrainian municipalities to facilitate policy implementation and evaluation. 

2.0 – The Ukrainian Pilot Project
The Initiative is but one component of the People’s Voice project. The People’s Voice is a comprehensive project aimed at building integrity at the municipal level in various pilot cities in Ukraine by building the capacity of local NGOs to monitor the delivery of local government services. It receives its core funding from the World Bank and the Canadian International Development Agency (CIDA) and is managed under the auspices of the International Centre for Policy Studies (ICPS). To date, the project has supported a number of initiatives in Ternopil and Ivano-Frankivsk including socio-political research, service “report cards,” public hearings, and the establishment of citizens’ advisory councils. Its fundamental purpose is to develop and sustain mechanisms that address the concerns both of local citizens and representatives of civil society. The NGO’s played a pivotal role in terms of the earlier initiatives in the two cities and they were expected to build on this involvement in the policy development component of the project.

2.1 – Policy training components 
As one of the leading policy analysis institutions in Ukraine, ICPS has developed a policy analysis training curriculum at the national government level. Senior government officials from the Secretariat of the Cabinet of Ministers of Ukraine and other central government agencies participate in on-the-job training. This initiative, coordinated by ICPS, is supported ad hoc by a number of countries including Canada, Sweden, and Great Britain, and international donors such as the International Renaissance Foundation. The ICPS training model is well established; it enjoys the support of the Secretariat; and can be adapted at the municipal level.  
The opportunity presented by the ICPS national level model has allowed the People’s Voice to initiate the Initiative beginning in March 2001. This is a response to the needs identified by the local administrations in Ivano-Frankivsk and Ternopil as part of the People’s Voice. As noted above, the Initiative is intended to improve the institutional capacity of the aforementioned city administrations and, specifically, to improve the delivery of municipal services. This reflects a two prong approach where the first prong entailed working with the NGO’s in an effort to support the elocution of the people’s voice in terms of what they desire through “report cards”, public meetings, and citizen’s advisory councils.  The second prong is to develop the government’s capacity to develop and undertake policy that reflects what the people articulate that they desire. Both components are important as the people’s voice is only as good as the government’s ear!
The primary goal of the municipal policy development initiative is to improve the overall quality of governance at the municipal level in Ivano-Frankivsk and Ternopil, and to help create opportunities and networks for increased public participation in the policy-making process. 
Senior local officials receive in-service training over a 10-month period and develop specific policy initiatives that are service-focused. The Initiative encompasses three key aspects: a) assistance in the development of sector-specific, service-related policy documents; b) further development of institutional capacity; and c) public policy training for local government officials.  
Taking into account that Ukraine is in the midst of a democratic transformation (and this transformation has long-lasting implications for the balance of power between various actors, namely governments, NGOs and citizens), there is often resistance to change within the public service: officials fear disruption, loss of influence or even job loss as a result of change within the administrative structure. Within the NGO community, concerns may surface over fear that their support networks or access points to the government may be affected. Similarly, citizens – if left uninformed about the transition process – may be left even more alienated from the government than before.  
A major gap exists between governments and citizens because of the transformation to democracy.  The government lacks both the skilled individuals and procedures to work effectively with the citizenry. Similarly, citizens lack the skills necessary to effectively evaluate and lobby their governments. There needs to be a public policy development process in Ukraine (i.e. weighing policy options and alternatives) that will respond to competing societal interests and thus develop policies that will benefit the whole of Ukrainian society.
Public officials lack both the means and skills necessary to critically evaluate their own policies, and to communicate effectively with citizens. Procedures for public consultation and outreach need to be developed that will address emerging policy issues.  Ultimately, advances in both the political and socio-economic realms depend directly on effective governmental institutions that can respond to, represent and protect the needs of citizens.
2.2 – Selection of the pilot cities
The selection of the pilot cities was undertaken by the People’s Voice Project and was based on several factors. First, it was important that the cities were viewed as progressive and receptive to innovations and reform in undertaking municipal functions and if possible the use of policy units and decision making via a policy process.  As these last two items are not well understood in Ukrainian municipalities, it was at least important to have a municipality receptive to reform.  This led the People’s Voice Project to examine cities where they had been working and where they understood the motivations for reform.  In addition, they desired municipalities that had an NGO that could participate and support the policy initiative.  As the early work of the project required working with NGO’s, the project selected NGO’s who were in these two cities.  As the People’s Voice Project had successful experience in working with NGO’s in Ternopil and Ivano-Frankivsk, both of them met the basic criteria for the next phase of the project. In addition, while discussing the possibility of undertaking this project in these cities, each city had a “champion” emerge within the city government who was very eager to take an active role in promoting, encouraging and facilitating the work that had to be done by the staff within the city. The “champion” is an important actor in encouraging and ensuring that innovative projects are carried out.  Consequently, the emergence of these people in each city was important to their respective cities being selected.  
The main criteria in selecting supporting agencies in the cities of Ternopil and Ivano- Frankivsk were: collaboration with local government, donors, and media; successful project implementation; and, cooperation with other NGOs that could support their activities in the future. The agency that was selected in Ternopil was the Ternopil Agency of Urban Development (TUDA). In Ivano Frankivsk, the agency chosen was the Urban Development Institute (UDI).  

2.3 – Implementation of the Initiative
Implementation of the People’s Voice in Ternopil began in April of 1999 under the auspices of TUDA and has been executed with the support of ICPS in Kyiv. The first stage of project implementation consisted of conducting surveys on the quality of local public services, carried out between April and June of 1999 by polling local citizens, public servants and business representatives from the city. As part of the local implementation of the project, a People’s Voice project advisory board was established, consisting of the mayor of Ternopil, local public servants and representatives of local NGOs. Soon after, the Ternopil NGO Coalition was founded with the purpose of improving the interaction of public organizations in the implementation process of some of the project initiatives. The members of this coalition (14 in all) took part in a training session to increase the effectiveness of their individual and collaborative work.  

The implementation of the People’s Voice in Ivano Frankivsk started at the beginning of 2000 with an identical survey to the one in Ternopil. The only difference was that the Ivano-Frankivsk survey separated out the service delivery issues related to the municipality and the Oblast (Region).  This survey was designed to identify the problems in the delivery of public services, and the needs of local citizens, entrepreneurs and public officials in improving these services. The results of the survey have been widely disseminated to local target groups and were also presented at the kick-off conference held in July of 2000 in Ivano Frankivsk, a conference that was attended by over 300 people.
After identifying the problems and issues that needed to be addressed in both cities, work began in order to unite local partners and resources, and to provide a solid local institutional base for the implementation of the project. The coalition of local NGOs, together with partners from the city government and project management identified major project areas in both Ivano Frankivsk and Ternopil. The spheres of activity in Ivano Frankivsk and Ternopil are shown in Table 1.

TABLE 1:
Identified Spheres of Activity in Ivano-Frankivsk and Ternopil

IVANO- FRANKIVSK | TERNOPIL
1) Strengthening local government capacity | 1) Strengthening NGO capacity
2) Strengthening local NGO capacity | 2) A gender audit
3)Education Reform | 3) Education reform
4) Utilities and communal housing (Zhek) reform | 4) Communal housing reform
5) Monitoring of public service delivery | 5) Local reforms of policy development for the municipal services initiative and communal payment centre

Launched in the spring of 2001, the Initiative helped apply practical policy analysis training to real work situations and work products in the project cities.  Public servants (who constitute the core of the intended policy capacity recipients in Ivano Frankivsk and Ternopil) received practical training in policy analysis to build their administrative capacity in developing future policies that respond to and benefit all sectors of society. The project consists of two distinct phases:
1.	policy analysis training and policy document concept development; and, 
2.	drafting and implementation of service-related policies. It is important to note that an evaluation was planned at the mid-way point of the Policy Development for Municipal Services project.  Continuance into the project was dependent upon the policy analysis groups’ successful completion of policy document concept paper. 
An agreement was made to aid the training process by creating six policy analysis units in each municipality. The focus of these policy units’ efforts would be determined by the Mayor’s office in each respective city (in association with areas where PVP work is focused, namely education, communal housing and transportation).  A select group of five or six public officials representing various government departments was brought together to develop new policies in each sector.  In Ivano-Frankivsk, policy units were created in strategic planning, budgeting, human resource management, communal housing, education and urban planning.  In Ternopil, they were created in strategic planning, budgeting, human resource management, communal housing, education and transportation.  The formation of these policy units represents the development of parallel policy development within these municipalities. The selection of the members of each group was undertaken by the leader of the initiative in each city that reported to the Mayor’s office.  These individuals were the “champion” or prime administrator of the project in each city and were responsible for keeping their policy teams on track and liaising with both the People’s Voice Project Office and the local NGO who was assisting with the development of the project.  It is interesting that in selecting the municipal employees to participate in the project, the municipalities selected both experienced senior officials along with some “rising stars” in the local bureaucracy.  There was recognition of the need to develop the skills of the younger employees and the leaders, department heads of tomorrow. 
The process was divided into several steps: 
1)	Introductory lectures focused on the process of policy formation, implementation and evaluation. As part of these lectures, each group was assigned to identify a pressing issue within their respective policy sectors. After forming a detailed policy problem, working groups were asked to identify alternatives or options for the resolution of the problems they had identified. Groups were then given an opportunity to present their recommendations to all participants.

The introductory lectures included:
Priority Setting: Describes the municipal priority setting process including the use of strategic planning
Introduction to the Municipal Policy Process: Introduction and overview of the municipal policy development process
These lectures were intended to be interactive in that several group assignments were given in which the groups had to report to each other and critique each others work. This approach was innovative for them in terms of a learning style.  Feedback from the group members was that they enjoyed this approach and looked forward to more of these types of group assignments/work with presentations in future seminars.  One person commented that she never before had the opportunity to make presentations in this type of forum. We had the impression that this was true for a significant number of the team members. Consequently, this approach also gave them the opportunity to gain experience for their presentation skills.  

2)	Consultations were held to define problems. After defining the problems during the introductory lectures, both international and Ukrainian experts conducted consultations for problem definition to clarify points not immediately clear to members of policy units.  This was undertaken over several days in each of the
respective cities municipal offices.  Meetings were held with each group  to focus on the policy issue/problem that they would address in their policy paper.  Care was taken to ensure that the problem was not too large in scope for the group to address given both the time frame and the human resources available given the size of the group.  Furthermore, care was taken to ensure the problem did not require technical expertise or studies that was not available to the group.

The Table 2 and 3  below summarize the policy problem/issue to be addressed by each of the groups in each city.

Table 2 – Ternopil Policy Analysis Units

| Policy Analysis Unit | Policy Document Output
1 | Strategic Planning | Long-term, strategic civic plan
2 | Budgeting | Revenue program aimed at examining ways to increase revenue and efficiency
3 | Human Resource Management | Hiring and recruitment policy (including staff training and promotion system)
4 | Communal Housing | Policy for privatizing housing maintenance services
5 | Education | Policy and procedures for public input into education reform
6 | Transportation | Transit system integration policy

Table 3 – Ivano Frankivsk Policy Analysis Units

| Policy Analysis Unit | Policy Document Output
1 | Strategic Planning | Communications plan for public engagement and exchange in shaping priorities and directions for the municipal strategic Development plan
2 | Budgeting | Capital budgeting program for housing-related infrastructure
3 | Human Resource Management | Hiring and recruitment policy (including staff training and promotion system)
4 | Communal Housing | Housing maintenance policy (service-focused)
5 | Education | Program to reorganize the school network, as well as policies and programs for the delivery of extra-curricular activities
6 | Urban Planning | Zoning plan and regulations for the central area of the municipality (encompassing a building permit and inspection system)

  3) Policy unit members were given home assignments. The assignment was to draft preliminary papers outlining their policy issues and the approach they planned to use to address these issues. These papers were to be submitted and then in turn translated for the international advisers who would be attending the next set of consultations in the project.  Some of the groups were very prompt in the delivery of the policy paper while several lagged behind and were quite late. This seemed to relate somewhat to the leadership and enthusiasm that existed in each policy group.

4)	Consultations with both international and Ukrainian experts concerning the preliminary papers. After the groups undertook the initial work on their policy issue a second round of consultations was undertaken with international and Ukrainian experts. These were held with each group in the municipal offices. The object of these consultations was to give some initial feedback regarding thee group’s initial work.  Two main purposes of these consultations was to provide feedback to ensure the group was on the right track in developing a policy paper; and to ensure that the policy issue was appropriately defined and understood.  It was important to ensure that the scope of the issue was appropriate for the time and resource parameters of this initiative.

5)	Further development of the policy papers required the trainees to learn additional analysis and development techniques. Various skills were taught during additional sessions for policy unit members. The first session, “Policy-definition, Principles and Capacity,” was designed to provide public servants with a forum for defining and discussing the relevance of policy analysis at the municipal level. The second session, “Problem Definition and Problem Structuring,” identified the challenges that public servants face in clearly defining policy problems.  In addition to these sessions, seminars were conducted on  “The Development of Policy Options” and “Policy Document Writing.”  Participants closely examined the various criteria applied to the development and evaluation of options for the resolution of certain policy problems as well as analyzed several draft policy documents to identify strengths and weaknesses in their composition. There was also a seminar in “Public Participation/Consultation and Communications”  in which the participants were provided information regarding the importance of public input into the policy process and the need to communicate public policy initiatives and decisions with the public.  This section stressed the need to incorporate both the inputs from this process and any resulting communication strategy with the public in the final policy paper.  These seminars, related assignments and discussions helped to equip these local public servants with the skills needed to begin their work in policy development, communication and document preparation.

After the delivery of these seminars, there was an additional round of consultations with the policy groups in each city with the experts to provide feedback as the policy teams worked toward the preparation of their final policy documents.

6)	Development of policy documents. All policy documents, unless otherwise specified in the following summary, should include the following sections:
a.	BACKGROUND
b.	SCOPE AND SEVERITY OF THE PROBLEM 
c.	DEVELOPMENT OF POLICY ALTERNATIVES
d.	ANALYSIS OF THE ALTERNATIVES
e.	RECOMMENDATION (S)
f.	PUBLIC CONSULTATION/COMMUNICATION COMPONENT
Although this component of the policy paper is listed last, it should be understood that the actual public consultation/communication component was integrated into the steps in the policy process outlined above. It is only listed last to emphasize that it was to be an important part of the process and consequently needs to be reported in the context of the policy paper.

2.4 – Outputs and evaluation
In order to evaluate and to measure the outputs of the project, the following criteria will be used:
•	Evaluation of the final papers by the groups, whether the paper was group managed or not;
•	Evaluation of the papers by both Ukrainian and international experts; and,
•	Evaluation of the formation process to enshrine how the theoretical skills are to be implemented by public servants during their work in public policy development. 

The most stringent evaluation of the development of the policy process can only take place in the longer run. This may include both an evaluation of the success of the policy that was developed and implemented, and the more modest evaluation to see if the policy was actually implemented.  As this is a pilot program and the development of policy planning in Ukrainian municipalities is in its early formulative stages, it would be logical at this point to only consider the more modest evaluation and determine if the policy is actually implemented.  As implementation requires an allocation or reallocation of municipal resources, the next section of the paper provides a simple framework of how both implementation and later evaluation may be undertaken through the use of appropriate budget procedures.

3.0 A Structure to Implement and Evaluate Local Government Policies

The pilot project focused on the development of a range of polices within the policy units created in the cities of Ternopil and Ivano Frankivsk. This did include but not explicitly focus on implementation and “ex post” evaluation after the policy had been implemented for a period of time.  The purpose of this section of the paper is to briefly outline an approach and structure that may be  useful to facilitate the implementation and on-going evaluation of municipal policies.  The approach outlined has been used by a number of municipalities in North America and elsewhere.  The expectation is that with careful adoption and assistance Ukrainian municipalities could adopt the approach or a variation.

In municipalities, as with all levels of government, the implementation of policy requires the appropriate allocation of resources to undertake effective implementation. This is best accomplished in an open and transparent fashion through the budget process, or via an adjustment/ amendment to the budget process when that policy or program is adopted or approved by the appropriate political body.  In the latter case, adjusting the budget, would be identified in the proposed policy paper or document where the “impact on the budget’ section of the policy documents would identify the resources that are required for implementation and if they are outside that existing budget allocations. 

Many Ontario (Canada) municipalities require that all policy reports to municipal council follow a specific format that includes the author, department head, or leader of the policy unit, identify the impact on the budget for each and/or the recommended policy option.

Once decisions are made regarding desired policies and programs, the question is how can the implementation of these policy decisions best be ensured throughout the budgetary process?  An additional question might be whether there is a budget format or technique that might be useful in assisting in the evaluation of policy decisions?

As some of the policies/programs/projects that reflect policy decisions will require capital expenditures for implementation and some that reflect programs will require operating expenditures, it is important to differentiate between these two types of expenditures in the budget process.  This is best achieved by having separate operating and capital budgets. This is the typical process and is in fact generally legislated in North American municipalities.  For the capital budget, discreet capital items that reflect approved policies are self-evident.  However, it is necessary to ensure that they are actually included are appropriately prioritized in the capital budget. 

With regard to the operating budget there are a number of different budget techniques or formats that have been used by local governments as the municipal budget process has evolved over time. The range of budget techniques stems from the most basic line item budget through performance and program budgets to Planning Program Budgeting Systems (PPBS), to Zero Based Budgeting (ZBB).  In practice a municipality may use a budget format that is a hybrid in terms of using elements of several of these approaches. It is possible to comment on these budgeting techniques in terms of the objectives that might be considered for any government budget process. These objectives may be identified as:
1)	to maintain financial control
2)	to provide useful information for efficient management decisions
3)	To improve program and financial planning.

Of the budget techniques identified above line item budgeting meets only the first objective, performance and program meets the first two, while PPBS and ZBB can meet all three objectives.  However for practical reasons, it is desirable to have most municipalities achieve the first two objectives via some application program-planning format.

Although program planning may be applied in a number of variations, it will also always focus on specific programs that will include a descriptions and objective of the program to be delivered, and include some form of performance measures for evaluation. As the objective of the program is stated, either determined by the manager responsible for delivery or the municipal council, it is possible to evaluate the performance relative to this objective.  As the programs will reflect specific policies or programs approved by council, with the stated objectives, the listing of the specific programs in the budget will ensure that they receive the attention of council in the budget process to ensure implementation.  Furthermore the inclusion of performance measures in the program budget will ensure evaluation of the programs at the end of year (budget cycle) to provide guidance to staff and council for implementation strategies and budget allocations in subsequent years.

An example of tying the budget process to policy implementation occurs in municipalities who have prepared a strategic plan. These plans tend to have action plans for the implementation of priority issues that often reflect specific programs.  Consequently, a program budget is an excellent vehicle to ensure that strategic plans are implemented rather than “remaining on the shelf” after their preparation.  In addition, using the program budget for strategic plan implementation also has the added benefit of evaluating the priority programs of the plan through the budget as part of the annual budget cycle.

Each year as the budget comes before council, it should have the evaluation measures from the previous year for each program in the budget, a number of which reflect priorities in the strategic plan, and other programs reflecting polices approved by council. Prior to the new budget allocation for these programs council will have the opportunity to evaluate the performance of these programs relative to the stated objectives. Consequently they can make new allocations from a position of knowledge, and if program objectives are not being met they can ask the program manager (staff person in charge of the program) what is necessary to meet the objectives.  This may mean a better process for delivering the program rather than simply allocating more money to the specific program. This demonstrates that the use of performance measures through the program budget may also give important feedback to the program managers. This may increase the efficiency and delivery of programs and policies, and most importantly ties the implementation and evaluation to the budget process.

There are a number of lessons that may be drawn from the use of program planning based on western municipal experience. First, it is important to have separate capital and operating budgets, to ensure that projects requiring large capital expenditures, and the use of debt financing, where municipalities have access to this approach, are considered separately from ongoing priority programs.  Second, the use of program budgets as a technique is very useful to focus on the program, especially priorities that reflect policy decisions. The program organization will focus on all of the funding aspects required for implementation. In addition, the use of the program budget builds in an evaluation process that should annually evaluate the programs.  As with any management/policy tool, it is important to develop the capacity to use the tool correctly and tailor it to the application in the context of the specific political jurisdiction.

4.0 Conclusion
 
Over the duration of this initiative to develop the capacity for local governments to create policy units in local government, twelve policy units were developed and nurtured to produce policy papers.  This involved approximately 60 municipal officials from the two cities, Ternopil and Ivano-Frankivsk.  This process required strong direction and guidance from the People’s Voice Project office, as well as strong leadership and from the “champion” in each municipal government.  The participating NGO in each city also had the opportunity to learn about the policy process and the development of policy capacity and the preparation of policy papers at the municipal level of government.  Although members of the NGO’s played primarily an administrative/coordination role, their members had the opportunity to listen to the seminars that were delivered and sit in on the consultations with the policy teams.

At this point it is not possible to evaluate all aspects of the success of the initiative. As stated earlier the ultimate evaluation is whether any of the policies developed in the policy papers are actually implemented and address the problems that they are intended to remedy.  As the implementation is crucial, the previous section in this paper provided an overview to how the budget process may be used to assist municipal policy/program implementation and evaluation.

However, as an intermediate evaluation, it is possible to evaluate the quality of the policy papers from several perspectives.  This may include the quality of the ultimate policy recommendation, and the quality of the actual policy paper in terms of the content and described process that the group went through in developing the paper.  Unfortunately, these papers have been submitted late apparently due to the impending elections in the Ukraine (March 31^st).  To date, nine of the twelve policy papers have been received, six from Ternopil and three from Ivano-Frankivsk.    As they have not been translated for the international advisors, their evaluation is not available at this time.

It is possible, however, to draw some conclusions and evaluate the process for the program based on the interim submissions, the response to the seminars and consultations that were undertaken throughout the project.  These conclusions are:

•	It is useful to select policy areas and problems that are currently being addressed in the municipality. These make the project and learning more real and have a sense of urgency.  Also, as the participants are undertaking this work partly on their own time, this approach ties into the daily work that they are undertaking. Furthermore, it may also provide input from other external policy experts as was the case with both the housing communal service and education reform groups in Ternopil and Ivano-Frankivsk.  Policy groups with problems that are currently on the municipalities policy agenda tended to perform better in the project.

•	Use many advisors, international and Ukrainian, to provide a range of ideas and approaches. However, be sure that only “one voice” is giving direction regarding the format and requirements for the policy paper.

•	Good “champions” within each municipality are an important part of the project.  This may be true for the project, but once policy units are established, the “champion” may have to be replaced by an internal policy advisor role where this individual will give advice on the policy process and the development of policy papers to the various policy groups.  The “champion’ role is important to keep the momentum of the policy teams going and to reinforce the needs and benefits of the program as well as help to ensure that deadlines are met and people participate in all aspects of the project.

•	When selecting the policy teams, it is useful to mix proven experienced administrators, with “rising stars” among the employees.  This will allow the interaction of experience with new ideas and reinforce a new generation of employees with policy skills and at the same time ensure that senior employees understand the process and its benefits.

•	Care should be taken not to mix members of the policy team two broadly from difference departments or areas, unless a predetermined policy issue requires a specific range of representation. For example one of the Budgeting/Finance policy groups had finance people from several departments rather than all people being from the finance department.  This lead to some difficulty in addressing a policy issue that all groups members could fully participate in terms of the analysis and development of alternatives.

•	The timing and sequencing of the seminars is important. The policy groups need to have the material/knowledge delivered to them at a pace and timing that supports their work on the policy papers.  The use of group exercises in the seminars helps to reinforce the material that will be used in the development of the policy papers.  Furthermore, the groups need to have timely feedback and consultations regarding their interim work and products.  This reinforcement and guidance is important.  For this project, the time and sequence was done fairly well, but it could be improved based on the experience in this pilot project.

Overall, it would appear that the project proceeded quite well with all participants being satisfied with the outcomes to date.  Obviously, more detailed evaluation will be undertaken once the policy papers are analyzed and in the longer run when any impacts of the policies recommended in the policy papers are implemented.  A final evaluation will be possible in the longer run to see if either policy unit or a policy process is used on an ongoing basis to assist in policy/program decisions in Ternopil and Ivano-Frankivsk.

BIBLIOGRAPHY

Aarrevaara, Timo, 1998, Councillors and Civil Servants in the Ukrainian Self-Government, Naukova Dumka, Kyiv

Amborski, David P., 1994, “Linking Community Based Strategic Plans to the Budgeting Process”, paper presented at the 24^th Annual Urban Affairs Association Meeting, New Orleans, March2-5.

People’s Voice Project, 1999, “Voice of the People Survey in Ternopil: April to June 1999”, International Center for Policy Studies, Kiev.

People’s Voice Project, 2000, ‘Report to the People’s Voice Public Opinion Survey: Ivano-Frankivsk April to June 2000”. International Center for Policy Studies, Kiev

People’s Voice Project, 2001, “People’s Voice Handbook”, based on the conference, “Citizen Participation in Decision Making at the Local Level in CEE Countries: Concepts and Practices, International Center for Policy Studies, Kiev.

People’ Voice Project, 2001, “Annual Report Overview:, International Center for Policy Studies, Kiev.

Graham, K.A., and S. D. Phillips. 1998, Citizen Engagement: Lessons in Participation form Local Government, Monographs on Canadian Public Administration No. 22, Institute for Public Administration of Canada, Toronto

Pal, Leslie A., 2001, Beyond Policy Analysis: Public Issue Management in Turbulent Times, Nelson Canada, Toronto

The World Bank, 1999, Intergovernmental Finance in Ukraine: An Agenda for Reform.