By adopting an empirical approach to the practice of data production, we shall attempt to answer a number of questions. How do the stages of data production common to Ethiopian EWS result in the consensus that is so crucial to the functioning of the food aid system? How are power relations affected by other factors arising from bureaucratic practices (routines that lead to the repetition of former ways of functioning etc.)? How do international and governmental actors form alliances? What is the interaction between the objectivity of the expert and the intersubjective nature of politics? And finally, what happens at the level of the village? By observing the way the EWS functions in the field Field study conducted in Amhara region between November 2002 and December 2004. at ground level, as it were, we can begin to understand the ways in which the work performed by experts intersects with politics in the production of statistics concerning food security.

This section focuses on the ‘multi-agency’ evaluations conducted in accordance with the calendar of the two principal harvests. The evaluations take place in two stages: in November, at the end of the main rainy season (meher), whose harvests are decisive in the country’s cereal production totals, and in June, at the end of the minor rainy season (belg), which completes the overall needs assessment.

Multi-agency evaluations are key elements in the cycle of aid allocation. Coinciding with both the agricultural and donors’ calendars, they lead to the annual DPPC appeal to donors. Published in late December/early January, this document establishes the needs which will set the aid machine in motion. Needs are subjected to a second revision in June/July. Multi-agency evaluations, which formalise the country’s beneficiary estimates, culminate in the official publication of results and statistics for each administrative level. Such evaluations are not exceptional measures; they are a component of a smooth-running administrative routine, but their results have a decisive effect on future food aid totals and distribution patterns.

A. DPPC CONTROL OF MULTI-AGENCY TEAMS

1. DIVERSITY WITHIN THE TEAMS

Multi-agency evaluation teams group together representatives of various ministries and administrative departments (agriculture, statistics, meteorology), NGOs (SCF, CARE, the Red Cross, etc.) and donors (WFP and USAID). As the aim of the evaluations is to enable all the system’s actors to arrive at a consensus, the teams spend most of their time in meetings. Whatever time remains is devoted to travelling from town to town - when they can observe the condition of the crops from the road - and to writing reports. The teams work under quite severe constraints which transform the evaluations into a twofold challenge, pitting them against time and uncertainty.

On four occasions, I accompanied teams operating in zones in the Amhara region - North Wollo, South Wollo, Wag Hemra and Oromya Zone evaluation visits depend on the harvest seasons. Meher evaluations exclude totally belg-dependent woredas. The zones group together the following woredas: North Wollo: Bugna, Dawent-Delanta, Gedan, Gubalafto, Haberu, Meket, Kobo, Wadla South Wollo: Kalu, Kelela, Kubaber, Legambo, Mekdela, Sayent, Tehuledere, Tenta, Wegde, Werebabu, Wereilu, Yama Wag-Hemra: Dehanna, Sekota, Zikwala Oromia: Artumafusi and Jile, Bati, Chefegoala and Daw.  The teams were multi-disciplinary and composed of five or six Ethiopian nationals who had trained as agronomists, meteorologists, statisticians, agricultural economists, geographers or cartographers. Teams were sometimes reduced to two or three members in other regions. Exceptionally, Judith Standford from the NGO Save the Children-UK joined the team in November 2002 to assist the regional DPPC. The names of the people mentioned have been changed.

Most of the team members were highly educated. Some had spent three or four years at agricultural colleges, while others were mathematics, sociology or geography graduates. Some had studied abroad. The DPPC experts who had begun their careers with the RRC had been trained in Eastern bloc countries such as Russia and Czechoslovakia; younger members had received their training in India. Some of the donor agency and DPPC representatives had studied in England or the Netherlands. Many of them had attended a variety of food security courses and seminars in Kenya.

Given the shortage of qualified personnel, the teams appointed in Addis Ababa by the WFP and the DPPC were often made up of people with little experience. The WFP tried to place an agent in every team. The Ethiopian government posted federal or regional DPPC managers to the most sensitive evaluation zones. Vacancies were filled by cadres from other departments (agriculture, statistics, meteorology), although very few personnel were available for this purpose. Finally, members of NGOs were able to choose their postings.

While team members were able to fall back on their technical training, they often lacked practical experience of evaluations. WFP and DPPC agents were clearly the most seasoned members. The former were full-time food security experts and conducted evaluations throughout the year. The latter were tasked with observing and supervising the distribution of WFP aid. In this capacity, they regularly visited local administrative offices in order to collect data, as did delegates from bilateral donor agencies. The representatives from NGOs I encountered were programme coordinators and thus relatively unfamiliar with the collection and processing of food security data. Of the experts on detachment from other United Nations agencies, most had acquired a good knowledge of the institutional environment through their work with the Ethiopian administration (the Ministry of Agriculture, for example), international NGOs and donor agencies.

2. THE PRIMACY OF THE DPPC

Given the heterogeneous composition of a team, the relationship between its members was not one of equals during the negotiations that accompanied evaluations. Members of international agencies seemed to feel more confident - evidenced as much by the working tools at their disposal as by their relative personal financial security. But their responsibility was diluted in a vast system marked by the cohabitation of international and national actors. Moreover, the control that international agencies exercised over their experts was slack compared to the grip the Ethiopian civil service maintained on its officials. The Gemgema, an annual internal assessment organised by the Ethiopian administration, offered a striking example of this permanent control. When this collective exercise took place, each public servant was subjected to a critique by his colleagues and had to criticise his own performance. An individual’s professional or personal failings were likely to influence the final assessment made by his superiors. To a large extent, the assessment determined the course of his career.

Finally, the federal DPPC experts were directly superior to those involved at regional administrative levels (woreda and zone), whereas donors’ representatives acted more as intermediaries between international decision-makers and the Ethiopian authorities. Having been given a ‘monitoring’ role, these representatives usually confined themselves to collecting data and transmitting it to the upper levels of the hierarchy, where the key positions were held by foreigners. They simply represented a diffuse external authority, and had no direct control over the internal functioning of public services. On the other hand, DPPC cadres belonging to the same Ethiopian administrative apparatus were constantly in the hot seat, and were subjected to relentless, inescapable pressure. Already seasoned by the routine practices that went on throughout the year and spurred on by a system that allowed them no respite or room for error, they were much more inclined to accept the pressure of negotiations with patience and determination!

As the above observations indicate, these diverse structural elements influenced individual behaviour and the way in which the evaluations were conducted. In practice, decisions and negotiations, as much within teams as at meetings with the administration, were guided chiefly by WFP and donor agents as well as DPPC delegates. The ‘multi-agency’ dimension boiled down to a face to face between the DPPC, WFP and, on occasion, USAID representatives. NGO members, who had little negotiating experience and did not have to fear for their careers, tended to stay in the background. They played a supporting assistant role, forming a passive retinue of almost silent witnesses. At meetings with woreda and zone committees, which were led by one or two individuals, they provided support during negotiations. When the principal speakers had run out of arguments, they could then offer an additional point of view.

When disagreement arose within an evaluation team, I noted that the DPPC government agents invariably succeeded in making their UN colleagues back down. They would wear them down by dint of harassment and/or arguments that called into question the value of their technical expertise. Whenever WPF representatives stood firm, they still failed to counter the DPPC view and were reduced to a minority by the rest of the team.

3. THE TEAM LEADER: A KEY ROLE

Because the teams were composed of people from many different backgrounds, they went through a ‘breaking in’ phase. Most of the participants did not know each other when the evaluation began; they gradually learned to work together through the allocation of tasks and balancing of skill sets, frequent briefings and debriefings, internal decisional meetings and the development of strategies for negotiating with the woredas. Much of this apprenticeship in teamwork was experienced through day-to-day social life.

If the team leader did not establish the basic rules at the outset, the team was condemned to improvisation and competition. The classic practices of withholding information - both technical (documents, data) and organisational (programmes) - as well as the imposition of decisions without consultation (visiting programmes, dates of meetings) accentuated the random nature of the process, the fragmentation of information and the uncertainties to which the experts were already subjected. The experts would discuss matters in sub-groups (based on car-sharing), but apart from the inescapable report-writing sessions, they would rarely get together to arrive at a collective decision. Plans were often finalised at the moment of departure, on the hood of a car with its engine running and doors open. Information was dispatched from the corner of a table in a restaurant or café.

These hasty exchanges often led to the duplication of activities (locating documents, arranging an appointment, writing a report) and the neglect of crucial data. But above all else, the lack of prior arrangements condemned the expert to a form of isolation, forcing him to approach woreda negotiations on an individual basis. The group did not form a unified body which had the ability to confront the local authorities. If one member was in conflict with the rest of the team, internal negotiations were seldom conducted beforehand but would occur after meetings had taken place, when beneficiary numbers had already been partially decreed. During woreda meetings, teams strove to erase their differences. If there was an internal conflict - between WFP and DPPC representatives, for example - the dissident minority kept its objections to itself and aired them afterwards, thus presenting woreda committees with the image of a unified team.

The team leader played a central role in maintaining the group’s cohesion. In consultation with the members, he defined the operational framework: the visiting calendar, appointments, allocation of tasks, data collection (pluviometry, harvests, livestock, prices, etc.), coordination of report writing and other elements. When dealing with external authorities, including those at regional level and in Addis Ababa, he represented the group, leading and concluding negotiations. It should be stressed that the driver’s role was often equally decisive in terms of the group’s cohesion. The driver sat in on the discussions that took place during rest breaks and facilitated the circulation of information when a team was divided. And even when divisions occurred, he remained a relatively neutral link in social terms. He wielded a certain influence over decisions concerning the visit programme and would sometimes block a woreda visit if he thought the roads were too bad.

The team leader’s role extended well beyond the operational framework and encompassed various aspects of daily life; he would influence the choice of hotels, restaurants and even menus. He orchestrated the social microcosm which would weld the team together until the evaluation was over. Evaluations were not simply a matter of dry meetings and report writing. They also involved meals, coffee breaks and trips to bars, activities which led to reciprocal invitations and afforded the opportunity for wide-ranging discussions and countless jokes. For some members, the evaluations were also a way to discover an unfamiliar region, visit magnificent churches (Lalibella, for example), or purchase a variety of local specialities such as woollen rugs or woven dresses from Wollo, Sekota honey and alcohol from Debray Sina.

Among the teams I accompanied, one individual, Ahmed (WFP), stood out for the efforts he made to fulfil the role of team leader. He would thus sometimes interrupt discussions with the woreda to seek advice from his team, and take a vote on matters such as kebele visits, the clarification of opaque data and the final beneficiary total. He always did his utmost to foster collective analysis, reprising and synthesising, verbally or in writing, divergent points of view, and reviewing matters with his team during meals and breaks. But despite Ahmed’s democratic approach, he was reduced to a minority when the team rejected his beneficiary estimate. Given the excellent 2003 belg harvests, he argued for a low figure, while his DPPC co-member, Zelalem, an RRC veteran, took the opposite view. The members belonging to NGOs and the Ministry of Agriculture all sided with the Ethiopian administration as represented by Zelalem. Ahmed was unable to control the negotiations, but nevertheless continued his attempt to unify the team until the very end.

At the time of another evaluation, Girum, a young meteorologist from Addis Ababa, was appointed team leader. His youth, unfamiliarity with the region and lack of experience of evaluation practices worked against him, and he soon found his role usurped. Although desperately trying to cling on to it, he was reduced to making the standard presentations at meetings and arranging the report-writing schedule, which no one respected. The lead was always taken by Zelalem, who fixed the appointments and directed and terminated discussions. As Girum was unable to unify the group, information-sharing was restricted, while Zelalem controlled decisions and negotiations. This was not achieved without friction and Mulugeta (WFP) attempted a series of manoeuvres. But Mulugeta, also new to his post and unfamiliar with the region, was no match for Zelalem. Girum and Mulugeta were toothless allies; they clarified secondary meteorological data and wrote reports in English, but were unable to influence decisions to any great extent. Given the absence of a legitimate team leader, the group fragmented, leaving decision-making power in the hands of a DPPC official!

A team’s composition therefore determined its ability to conduct negotiations with woreda authorities. In most cases, I noted that fragmented teams allowed one or two decision-makers to take the reins, with DPPC cadres being most likely to clarify matters and reach conclusions.

B. BUREAUCRACY AND THE DOMINATION OF FIGURES

1. IN THE OFFICE

The evaluation process was subject to the geographical hierarchy erected by the Ethiopian administration. At each level - kebele (commune), woreda (district), zone (department) and region - the beneficiary totals calculated by local DPPC agents had to be validated by a local committee composed of agents from services such as agriculture, livestock, health, water and the rural economy. Besides respecting an immutably centralised order that stretched from Addis Ababa to the woreda, evaluation teams also had to deal with these multi-disciplinary committees. The organisation of meetings was a particularly onerous task as each stage required the presence of all the interlocutors, many of whom had other priorities or were committed to other meetings.

These meetings meant that the teams were compelled to travel back and forth over a three-week period, collecting information over the course of briefing sessions at each stage and then re-ascending the levels of the hierarchy, negotiating and gradually consolidating the figures. Given the number of meetings, it was not long before their work became extremely laborious and tedious, a situation compounded by the fact that information was presented again and again according to a stereotyped pattern featuring the same check list of questions and often the same wording. Once the routine had been established, the enthusiasm they had displayed at the initial meetings was replaced by profound boredom. Discussions with woreda representatives were punctuated by deep sighs and irrepressible yawns. Some experts would simply slump in their chairs and let their minds wander, or leave the room, allowing their most seasoned colleagues to cope with the trial of talking to DPPC officials. However, they managed to regain their taste for life during the report-writing sessions that marked the end of the evaluation. By resorting to irony as they discussed the litany of data they had had to endure, and would mimic a machine automatically parroting its message: “because of premature sporadic breaks in the rain, the harvest was completely ruined, etc.”

The implacable routine was exacerbated by the constraints of time. Even before the core issues came up for discussion, the agenda had to be adjusted to take into account briefing and debriefing sessions with every team at woreda and zone level. Travel between woreda capitals (which were often several hundred kilometres apart) and the time allotted for writing reports also required adjustment. The limited availability of local officials - who were very often away on a visit, at a meeting or attending a workshop - made it even more difficult to organise meetings with the representative committees authorised to provide the data necessary for the completion of the evaluation forms. There were times when the data had still to be validated by the local committee, and required further behind-the-scene adjustments before negotiations could begin. The teams were often condemned to long periods of idleness, hanging around in a corridor while all the experts gathered in the meeting room, while extra chairs were collected from various offices, or while the key to the office containing the crucial dossier was retrieved from a secretary who had gone on leave.

Finally, the task rapidly degenerated into an exhausting hunt for elusive interlocutors and partial sets of figures. As a consequence, other ways of information-gathering - visits to markets, interviews with peasants - were minimised. As there was little input into decisions from other information sources, the main evaluation framework was reduced to bureaucratic exchanges in offices, conducted principally with reference to written reports and sets of figures.

2. FIGURES

As a meeting progressed, each expert from the local committee provided detailed descriptions of meteorological events, the quality of various crops, the different types of damage sustained by crops, the condition of livestock, the health of the population, etc. Such presentations involved the reading aloud of reams of statistics, all of which were scrupulously copied down by every member of the evaluation team: there were figures for ploughed, sown and harvested areas, quantities harvested by crop type, losses by damage and crop type, livestock totals, etc. These inventories were of course detailed down to the last decimal point before being transformed into percentages.

The totals were put together at kebele level by ‘development agents’ Development agents were technicians and worked for the Ministry of Agriculture; they had received from two to six years of training. Based in the kebeles, they represented the initial technical interface between government and peasantry. They were tasked with “organising” the farmers by providing advice and training, supplying materials such as insecticides and fertilizers, demonstrating improved techniques, etc. They also supervised the Food for Work programme, participated in kebele committee meetings and collected agro-pastoral information, which was then sent to the district authorities. and sent on to the woreda (district), where they were sometimes amended by the observations of the woreda experts in the field. The compilations were transcribed into reports and tables, usually printed but sometimes existing only as rough copies, masses of sheets taped together into huge parchments, covered with alterations and plastered with correcting fluid, the traces of earlier revisions.

Multi-agency evaluation teams based their woreda beneficiary estimates on these compilations of agricultural data. Direct investigations at kebele level were highly unusual. When meetings were concluded, the figures for the woreda were reprised and incorporated into evaluation reports destined for the zone, the region and, eventually, Addis Ababa. Apart from isolated, localised field visits and an overview gleaned when travelling from place to place, the information the teams had collected was presented in written reports. The role of the teams was to justify and validate the contents of the woreda reports and then incorporate them into further reports which would eventually be synthesized in the form of the country’s annual appeal to donors.

In the main, the reports were composed of figures. The narrative sections often simply duplicated the wording of the previous year’s reports, amending it according to the agricultural and meteorological situation and sometimes adding an entirely new item of information. In other words, expert activity was centred on the collection and validation of figures, the compilation of tables and the filling in of forms.

During the November 2003 evaluation, the teams were asked to provide separate figures for male and female beneficiaries. But calculated on the basis of agricultural data, the overall beneficiary total did not reflect the physical reality of a village’s population. However, once the beneficiary estimates had been finalised, Girum, the team leader, thought it would be appropriate to base the calculation on the population statistics for each woreda. In effect, he managed to fill the male/female beneficiary table in the final report with an incredibly detailed set of data. When I expressed my astonishment at this procedure, he hastened to reassure me: “But you can trust me, I’ve got a mathematics degree!”

C. THE GULF BETWEEN EXPERT AND PEASANT

Meetings at woreda level constituted the only window on rural reality. While the view was somewhat obscured by the piles of reports and columns of figures and percentages, it was possible to zoom in on specific points by rapid visits to certain ‘representative’ kebeles in the company of a woreda expert. Damaged fields could then be inspected and a comparison made with other, more fortunate kebeles. Such visits sometimes provided the opportunity for fleeting meetings between peasants and experts.

But interviewing peasants seemed a superfluous activity in a working environment of office meetings punctuated by the pursuit of figures, the compilation of data and the writing of reports. If a meeting with peasants was agreed, it had to be brief because “we’ve spent so much time with the authorities that we can’t spare more than 15 minutes!” It was a formality the teams were reluctant to observe, and had to be dealt with as quickly as possible. Indeed, interviews with peasants were completely overlooked during some woreda - and even zone - visits. There always seemed to be an obstacle - lack of time, fuel shortage, poor roads, etc. - to site visits and meetings with peasants.

Contact with local populations was invariably the result of hurried chance and occurring when the vehicle stopped or passed people at the roadside or at work in their fields. At times the team would simply remain in the car, lowering the window to exchange a few words. If a meeting with a peasant required any walking, it was a major obstacle. In November 2002, Judith (SCF) encountered numerous problems when she wanted to visit Ziquala woreda. It was extremely hot, the area was barren and stony, and its villages were very difficult to reach by car. As Judith insisted, the other team members agreed to attempt the expedition, but the track petered out after a 15-minute drive. Urged on by Judith, DPPC officials Bekele and Getu walked the remaining several hundred metres to a hamlet. The others, Anteneh (UN) and Fissaye (USAID) took shelter under the shade of a thorn tree some twenty metres from the car, where they waited for anyone who happened to pass by.

The few contacts between experts and peasants were brief and anonymous. Most often, a compact team of four or five experts in a 4x4 would drive up to a group of peasants going about their work; disdaining introductions, they would start firing questions at them. Getu, an Amhara regional DPPC official, informed me that “introductions are pointless. It’s not worth the trouble; they know perfectly well why you are there.” On occasion, the experts did not even know the name of the locality they were visiting. Words were exchanged as they stood under the sun, two or three metres away from the peasants. At worst, these face-to-face encounters turned into systematic and fruitless interrogations that simply resulted in evasive, contradictory or implausible responses. This tended to provoke the anger of certain experts like Ahmed (WFP). Ahmed took exception to what appeared to be deliberate incoherence and the fact that his team-mates did not seem concerned by it: “No, I’ve lost all interest in meeting any peasants whatsoever! Look at that one over there! He tells me the harvest is pathetic, yet he’s standing in his field with the corn up to his waist!” This inability to conduct proper conversations with the peasantry meant that no information could be gleaned, and it clearly reinforced the tendency to dismiss interviews as totally pointless exercises.

At best, an expert might manage to get away from the group and conduct his own interviews. The farmer could then offer his own views on the area’s rainfall and harvests, show the expert some samples and take him to a neighbouring field to compare the quality of ears or soil. I met only one such expert, Bekele, a passionate agricultural technician from the Ministry of Agriculture, who believed he could arrive at well-founded conclusions by supporting his own observations in the field with information gathered from peasants. Although somewhat autarkic in his dealings with his team-mates, his contact with farmers was characterised by warmth and loquacity. He would introduce himself, shake hands and chat intimately, exchanging points of view. Bekele was clearly annoyed by the importance placed on office meetings, particularly as his overly nuanced views went over the heads of his team-mates and were not taken into account.

While the discussions between experts and peasants were virtually meaningless, they were typical of the relationship between urban expert and peasant, with the former subjecting the latter to a barrage of questions. They also revealed the strategies of avoidance and resistance employed by the peasantry, whose responses were incoherent or riddled with implausible exaggerations. And they illustrated quite clearly the gulf that existed between the expert’s world and that of the peasant.

D. SUBJECTIVE METHODOLOGIES

1. THE EXPERT’S CONFUSION

Every evaluation team expert, including the DPPC, WFP and FEWS organisers I met in Addis Ababa, acknowledged that the methodologies employed were ‘subjective’. Methodological subjectivity was the main complaint voiced by experts at briefings, meetings and interviews. A recurrent discussion topic, it was dealt with fairly rapidly for there seemed to be no alternative to it.

Thus in November 2002, at the initial reception organised by Aderaw, Chief Commissioner of the Amhara regional DPPC, Ahmed (WFP) wasted no time in expressing the general dissatisfaction with the situation: “No clear methodology has been provided by the federal authorities. What methodology are we going to use?” The reply indicated that the teams would simply have to cope: “An early warning committee is currently studying a new methodology, which should be tested in Wag Hemra. It is based on rather complex calculations and formulae! At present, the methodology is empirical, according to the individual’s experience, rather subjective. But for the time being, the usual methodologies should be used, the ‘balance sheet’ will always provide some indication.” Ahmed politely pointed out that the balance sheet was not entirely reliable. Another WPF expert tried to offer him some comfort: “There have been many attempts to obtain data at grassroots level, but they have failed. There is no way of arriving at a precise number of people. There is no magic formula; it has been that way for twenty years. That is what makes foreign donors suspicious; they think the figures are manipulated!” The regional DPPC Commissioner did not flinch at this, but simply added: “These methods should be taken very seriously.” In short, make do with what you’ve got but bring us results, even if the method prevents you from working properly.

We encountered similar lucidity at zone level. In North Wollo, Moges, the local food security expert, said that “Proper targeting of beneficiary populations is a problem ... The method we use is too subjective!”

In private discussions, the experts’ remarks were always shaped by unanswered questions: “Over the past year, I’ve found that the use of the ‘balance sheet’ is problematic. Should we use it in its entirety? It relies too heavily on personal judgement concerning elements such as livestock ownership and individual incomes.” Similarly, “Reliance on such scenarios will not produce a true picture of reality. It is simply an indicator, a technical supposition that does not reflect the circumstances in each district, especially when there are differences from one kebele to another.” Ahmed (WFP) described the methodology’s subjectivity in the following terms: “As a methodology, it has led to many errors. The evaluation depends on the field experience of the person conducting it. For example, if the expert comes from the city - and if his knowledge is theoretical - he will not in practice be able to compare the data with the quality of the crops, or estimate yields or understand the peasants’ survival mechanisms. His conclusions could be the opposite of those of an expert from a rural background.” Beletu, a DPPC expert from Addis Abeba, indicates his unease: “the estimate of crop losses are very subjective measurements. Sometimes we measure the yield per square meter, but we are dependent on the data from the MOA agents and the information we collect from the peasants”. Zelalem (Amhara regional DPPC) categorically deplored the “lack of a proper methodology. Anybody can say that his own methodology is the best, since there is no valid methodology!” Zelalem was a fatalist and had become a master at adjusting data, which he did with a good deal of tact and, dare I say it, professionalism!

2. AN ELUSIVE METHODOLOGY

In most cases, the experts adopted an empirical approach, basing their calculations on a combination of the FAO Food Balance Sheet and socio-economic data. The Food Balance Sheet was largely concerned with the annual agricultural deficit linked to losses and the annual total of food aid needed to cover that. Quantities were expressed in quintals per annum and then converted into beneficiary numbers, using a coefficient that fluctuated between 12.5 and 15 kg of cereals per person per month. However, other indicators were sometimes taken into account. In November 2002, Getu (regional DPPC) was training with Judith (SCF). Lacking a computer, he worked with his calculator and checked the calculations by hand several times in his notebook. Getu relied on the percentages provided by the authorities in order to subtract from global needs estimates the food generated by livestock rearing and general commercial activity. In the course of the 2003 and 2004 evaluations, this type of methodological combination spread to other teams, which deducted from the aid totals the proportion of food that was accessible as a result of non-agricultural incomes.

At the end of 2004, despite different trials of new methodologies, the Amhara regional authorities had failed to adopt a new standardised approach: neither the propositions that had been produced after nearly three years work by an Addis Ababa-based technical committee composed of WFP, DPPC, FEWS and SCF representatives and a foreign consultant, nor the possibilities arising from SCF-UK’s household economy model, nor the WFP’s techniques for visually estimating yields brought any radical change.

The SCF Household Economy Survey, tested in Ethiopia in 1992 and later refined, was adopted in one region in 2005, but did not replace the standard methodology in use in the rest of country. In accordance with their ‘capacity building’ strategy, SCF-UK representatives had been trying to persuade the DPPC to adopt this method for more than ten years, but only a few laborious steps had been taken. The support of consultants working with the Addis Ababa technical committee and the Amhara regional DPPC teams (Judith) had not led to a new investigation manual or even to the methodology’s application by the regional DPPC. SCF’s development of a database on the local economy and agro-pastoral situation (the ‘baseline’, to use the jargon) in certain parts of the Amhara and Ogadan regions had as little effect on customary practices as the practical training given to various WFP and DPPC managers. At the end of 2004, the commitments undertaken by the head of the early warning department in Addis Ababa remained a dead letter, allegedly due to the hurried departure of the SCF expert tasked with writing the definitive general training manual for DPPC agents. Only the Southern Region (SNNPRS) had opted for the methodology and had begun creating an economic database (baseline) for each agro-ecological zone.

DPPC and WFP managers tended to justify resistance to change in a number of ways, citing technical considerations, the logistical impossibility of conducting simultaneous surveys in the villages during multi-agency evaluations, the difficulty of determining to what extent a few kebeles were representative of a vast agro-ecological zone, the complexity of computerised calculations and the subtle hand required for extrapolations and final calculations. They also mentioned the risk of manipulation during interviews by peasants and, finally, the cost of the methodology, which required regular updating of the database. WFP manuals setting out methods for measuring yields remained in offices and were still not being used for the final evaluations in November 2004.

The experts’ long wait for a new and more objective tool was exacerbated by their disappointment over the methodologies on offer, which were still tainted by subjectivity: “It’s not worth changing if we are faced with the same problems of subjectivity! It’s still a matter of resorting to the same old tricks!” They regarded the establishment of a reliable methodology as a kind of mirage, and were forced to use subjective methods, to make sense of fragmentary data and tolerate generally chaotic working conditions. Ahmed took a philosophical view of the rules of the game: “You just have to cope with it! In such circumstances, everyone has to develop his own strategy for obtaining the best information.”

3. THE ART OF EVALUATION

a. The jumble of data

The expert therefore had no data that he himself had derived directly from measurements in the field. He was forced to base his decisions on second-hand figures compiled by the woreda offices. Given the absence of information sourced directly from the field, he had no quantitative tool which allowed him to compare the data collected with the reality.

Initially, some of the preliminary data was passed to the team leader at a briefing in Addis Ababa. This session took place in the DPPC’s vast meeting room, where the enclosed atmosphere, red curtains and neon lighting rapidly set the scene for the soporific boredom of the coming exercise. DPPC, WFP and FEWS representatives provided general information about the national situation and highlighted the zones at risk. This was followed by a theoretical outline of working methods, which the participants were invited to comment on. The list of methods and approaches came straight from a food security manual and it soon became apparent that they bore little relation to current field practice. The meeting’s organisers commented on the technical reference data and the working calendar, both of which formed part of a folder known as the working ‘kit’. The folder also contained a compilation of data from previous years. Unlike the previous year’s DPPC evaluation reports, these documents were composed of raw data and contained no analysis. The ‘kit’ was chiefly composed of indicative data for each district.  Rain curves per district, measured by satellite over the past six months. - District maps indicating altitude and rainfall - Population figures for the current year (based on the projection from the 1994 Ethiopian Statistics Centre census). - Tables of official data compiled over the previous ten years: beneficiary population per district; different crops per district (cultivated areas expressed in hectares) and annual yields (quintals); quantities of fertiliser distributed per zone. - Economic activity percentages of the population of each woreda. - General statistics concerning the average size of a family, the average length of dependency on food aid (six months), the average area of a family holding (0.81 ha per family), the average number of animals per family (0.81 oxen, 0.76 cows), etc.

The initial briefing was followed by visits to the zone and woredas, where the experts collected the official annual data relating to expected harvests, ruined crops and the populations affected. The district-based data sets for each type of harvest were systematically calculated to the nearest decimal point! Estimates were given in terms such as 18,626 hectares of cultivated land and 175,676.5 quintals harvest yield, etc. Losses were expressed in percentages, which were used to calculate percentages of the percentage (“of the 87% planted, there was a loss of 63%”). The data specified the losses caused by various types of damage (drought, hail, weeds, floods, etc.), the percentage of livestock that had migrated because of drought or the number of dead animals. However, data relating to morbidity (malaria) or malnutrition was always vague, e.g. “there are many cases of malnutrition in this or that kebele”.

The expert was therefore confronted with a mishmash of data, some of it over precise and some of it imprecise, which in most cases had been synthesised at woreda level, without any detailed data concerning the kebele level. His work consisted of piecing together the fragments in order to reconstitute a coherent picture of the reality of the situation. He therefore had to think in relative terms, comparing the agricultural situation and beneficiary estimates with three principal references: scenarios of the agricultural situation giving the range of beneficiaries, data from the previous ten years (agricultural deficits and beneficiary totals) and his own observation of the agricultural landscape.

b. Indicator scenarios and data averages

The working kit handed over in Addis Ababa contained indicative woreda beneficiary figures based on three indicator scenarios: ‘Best-Middle-Worst Case’. These sets had been established the month before, when multi-agency evaluation missions had conducted joint WFP/DPPC visits to most of the woredas. They served as a guide to the expert’s decisions and as an authoritative argument (the Addis Ababa estimate) when negotiations became difficult. Bolstered by more general information, these data sets provided the initial decisional framework for each region and defined the limits that neither the woreda authorities nor the ‘multi-agency’ evaluation teams were supposed to exceed. In its own way, the data production process was a striking phenomenon; functioning in circles, it duplicated the same improbabilities at different levels, and finally validated decisions that had already been largely ratified.

At woreda level, experts received harvest and deficit data as well as an initial beneficiary estimate. All negotiations began by comparing this raw data and the ‘agricultural deficit/number of beneficiaries’ ratio with that of the previous year or with the averages. In general, the experts took as their reference point a ‘normal’ situation derived from the average over previous years, the beneficiary average being one example. Discussions with the authorities almost always began by comparing harvest differentials from one year to the next with corresponding beneficiary differentials. Any disproportionate figure required justification; the teams would then resort to statistics and launch the negotiation process.

However, ‘normal’ data was still open to dispute. The members of the team responsible for the 2003 belg evaluation could not agree on the definition of a ‘normal’ situation and engaged in interminable discussions: “Should the average be calculated over five, eight or ten years? Should we discount the years when harvests were poor or exceptional? What is a normal situation? Do we take the figures that reoccur most often in the last five years? Would that be the average?” Zelalem later explained that “there isn’t enough precise data to define a baseline. We did not have the time to compile the baselines at Bahir Dahir [capital of the Amhara region].” According to Bekele (WFP), “The notion of a normal situation is meaningless. Addis Ababa and the Region asks us to take a normal situation as our reference point, but in practice we do not have a specific reference point.”

Whatever the case, historical precedents played a major role in determining the final data. But comparing the current situation with historical averages implied that all previous exercises were reasonable and realistic, that the statistics, or rather the decisions based on them, were sound. Given the considerable turnover of team members (only DPPC and WFP managers held permanent postings), the data from previous years was rarely challenged, for no one knew under what conditions the figures had been arrived at. This procedure served to perpetuate a system that constantly recycled its own references and data and could also function as a closed circuit, totally divorced from reality.

c. An empirical survey of the agricultural landscape

In order to extricate himself from the stifling bureaucracy, the expert had only one tool at his disposal: empirical observation of the agricultural landscape. Fissaye (USAID) put it bluntly, telling me that the woreda authorities “may well have their own criteria, but I don’t take that into account. You do what you have to do. What we need is a baseline, a line we can follow.” Other experts proceeded in similar fashion. The creation of a ‘baseline’ required being in a position to estimate the inventories of a woreda or kebele. In practical terms, this meant conducting a visual estimate of crop and herd conditions when driving along the roads. Zelalem modestly described his own method: “When travelling from one place to another, you can compare crops even if you’re not a professional in every domain. This enables me to memorise the condition of the crops and be prepared for discussions with the woreda authorities if they try to dramatise the situation. Some experienced agronomists can even judge production per hectare at a single glance, just by the colour and density. I simply estimate the percentages of losses. When travelling in the car, I can see the continuous line of fields bordering the road, and I look for alternations in the quality of the crops. I count the good and bad crops; that gives me an estimate which I then convert into rough percentages. If you want to observe the fields from high ground, you need binoculars.” Following a half-day visit to two problematic kebeles during the belg in June 2003, Ahmed’s team was able to agree on percentages concerning crop growth and losses, figures that were essential to the negotiations.

This type of crude estimate enabled teams to define a baseline from conditions in what were thought to be the worst-hit woredas and kebeles, thus establishing the maximum level on the scale of gravity. According to his experience, each expert was able to create his own scale by referring to previous years or to other zones which had been recently visited. In my view, the personal method based on visual comparisons manifested the originality of each expert’s individual know-how. As we have seen, he was obliged to refer to pre-established scenarios and ‘classify’ situations in order of priority. The expert had to hone his observational powers and analytical skills in record time in order to respond to this categorisation of needs. The more experienced he was, the more he was able to commit different situations to memory, compare them and classify them in relation to others. As the constraints of the job forced him to travel from place to place without gaining much information, the ‘baseline’ constituted a personalised tool, perhaps the most appropriate one at his disposal. At least it was the most honest tool, given the prevailing tendency to distort figures. And furthermore, the expert could adapt it as the evaluation progressed, for each visit provided a new insight into the agricultural landscape and condition of the crops.

E. FORMAL NEGOTIATIONS

Woreda and zone were the two main negotiation frameworks. The aim of meetings at woreda level was to agree on a global number of beneficiaries in the light of agricultural deficits. Following the interminable recital of data relating to the local situation - a preamble which enabled the protagonists to sound each other out - the meeting focused on adjusting the final beneficiary figure. At this level, power relations usually favoured the evaluation team. Teams managed to gain the upper hand through technical arguments, their grasp of the methodologies, the use of comparative data and, sometimes, through the resort to authoritative arguments. If the woreda committee refused to accept its figures, the team would have to persuade the zone authorities. The experts did their best to conclude matters at woreda level, knowing that negotiations with zone authorities, who generally sided with the district, would prove more difficult. The woreda therefore held a trump card, for an appeal to the much more politicised higher level would result in the experts losing their advantage.

1. NEGOTIATIONS WITH THE WOREDA

The November 2002 negotiations with Wadla woreda provide an illustration of the main types of argument.

1.1. Comparative arguments

An initial challenge of the figures was based on the comparison of recent data with the averages (figures for the previous year, figures for good and bad years), and with the projections contained in the Addis Ababa scenarios. A team which had already started the evaluation and had fixed the results in other woredas, would took into account the previous agreements which, as we have seen, defined the ‘baseline’, the experts’ evaluation scale. This evolving approach accentuated the relative and subjective nature of the results. The scale of the readjustments and the teams’ inflexibility increased as they gained confidence and accumulated victories in the negotiations. Comparisons enabled a preliminary trimming of the figures and the destabilisation of local committees, for they highlighted the ‘unjustifiable’ incoherence in the latter’s estimates. They also paved the way for debate on the data and methodology.

At the opening of negotiations in Wadla, Getu (DPPC) made an initial calculation before getting down to details; the result automatically undermined the beneficiary total of 20,000 advanced by the woreda. He immediately deducted 20% from the food needs total, arguing that the food estimates showed a surplus in relation to the previous year. With a few swift clicks on his calculator, he showed that the harvests met the population’s needs and exclaimed “We might even have a surplus here! If this is the case, there is probably no need for food aid!” There was a deathly silence in the room. The negotiations were under way.

1.2. Methodological arguments

By taking socio-economic factors into consideration, the ‘new’ methodological approach also allowed evaluation teams to dispute woreda figures, which in most cases were derived from the Food Balance Sheet basic calculation (the cereal production/local consumption estimate). The coefficients facilitating the transition from agricultural deficits to beneficiary totals were subject to negotiation, and some woreda experts were still using the old RRC coefficient The coefficient enabled the conversion of agricultural deficits into beneficiary totals. The WFP coefficient adopted by the DPPC was 1.89; the RCC version was 1.81. Moreover, Wag Hemra officials used another argument based on RRC calculation methods in November 2003. They sought to reduce the beneficiary total for Ziquala woreda, where needs estimated according to the Food Balance Sheet were reduced by 25%. Zelalem later explained that the “3/4 and 1/4 rule was inherited from the RRC, which would never initially countenance a 100% loss but would commit itself only to covering 75% of a deficit!”

which inflated beneficiary figures! In addition, the evaluation teams insisted on taking other peasant activities into account and expressing them as percentages, so that the fraction of the population whose resources did not depend on agricultural activities could be removed from the beneficiary total.

Local representatives, unable to counter methodological arguments beyond their competence, were forced to give way to the expert’s knowledge and selection techniques. On the other hand, the percentages for each non-agricultural activity, although based on data provided by woreda statistics departments, were also subject to bargaining. The introduction of the socioeconomic approach was a gradual process and was largely ignored by the woredas in November 2002. In November 2004, most of the woredas were still not using it systematically, and were leaving it to the experts! Although the method had gained a measure of acceptance, it was still necessary to negotiate the proportion of non-agricultural activity that contributed to household food resources.

The Wadla negotiations focused on the percentage of food needs covered by casual labour and small businesses. The authorities’ figure rose to 1.5%, while the SCF estimate for impoverished populations was 40%. The woreda representatives disputed the figures in the SCF survey: “Poor populations cannot derive 40% of income from day labour, that threshold cannot be reached because there are no job opportunities in the area.” Getu accepted their point and both parties finally agreed on a figure of 10%, which enabled them to arrive at a total of 10,000 beneficiaries (a ‘coherent’ figure supported by DPPC baselines and scenarios).

1.3. Technical arguments

Technical discussions were based on the examination of registers and archives and a painstaking analysis of figures and information. The experts thus needed a more specialised knowledge of agricultural practices and a precise understanding of the way the woredas produced their figures. Agricultural assessments for each type of crop were recalculated, often by modifying the yield per hectare (upward or downward revision), which sometimes required further negotiation over yield differentials. Types of loss were also discussed. Losses linked to weeds were not taken into account, for they were regarded as the result of “the peasant’s idleness” rather than as a natural calamity. The kebele was studied in whole or part on a case by case basis, as was the inventory of damage by crop type. The evaluation teams estimated loss percentages by observing roadside conditions as they travelled through the area.

In Wadla, Getu overcame the woreda committee’s remaining resistance by correcting a “misunderstanding” in its reasoning: the inclusion of the 3,000 people dependent on the minor rains of the belg. He explained that there was “no reason why belg-dependent populations should be included in the number of meher-dependent beneficiaries”. Everyone agreed on a final total of 7,000 beneficiaries and parted on friendly terms. The negotiations had been ‘fair play’.

1.4. Hierarchical arguments

When an evaluation team had run out of ideas, it resorted to the higher authority argument. During the negotiations in Dehanna in November 2003, Befekadu (CIDA) demolished the last of the woreda’s objections by comparing his own estimates with the ‘Addis Ababa scenarios’. Concluding that a beneficiary total of 31,000 was too high when compared to other woredas, he said, “OK, it’s a lot in comparison with Kobo and Habru, where the maximum is around 12,000-15,000. Especially as the problem here is chronic and the rain curve is good. Medium-altitude zones, like most of those in this area, have experienced favourable conditions.” The woreda representatives kept returning to the heavy damage caused by hail and the subsequent losses. Befekadu switched to methodology: “What method did you use to arrive at 31,000?” The woreda DPPC expert referred to the Food Balance Sheet. Befekadu asked: “Did you not take other incomes into account? Don’t you have the calculations for sharing out the incomes of the woreda?” He was told that they did not receive the form and that the DPPC committee did not have the data to hand. Befekedu claimed that the figures relating to non-agricultural activities were too low: “The North Wollo woreda average for other activities is about 30-40%.” The woreda experts justified their findings: “But there is no market here, as it is a remote area and there is no way for people to make money.” Befekadu pointed out that the woreda was renowned for the value of its honey. The local committee stuck to its guns: “But we’ve already told you that there has been a lot of damage caused by pests, etc.” Befekadu persisted: “Tell me about the scale of these losses again ... A hundred thousand or so, OK, that’s nothing! We need to be reasonable and reconsider this figure! Yet again the Dehanna figures are high, whereas they’ve fallen everywhere else! You know, at national level, the threshold is five million. Moreover, there are other possibilities for the peasants, like ‘cash for work’ programmes.” The woreda experts refused to budge but finally had to admit defeat: “But you know perfectly well that the soil here is not very fertile. Ok, we accept that we did not take other sources of income into account. We are prepared to reduce the figure.” Befekadu breathed a sigh of relief: “Ah, that’s good. In Kobo, for example, it dropped from ... how much was it, remind me? Ah yes, from 40,000 to 15,000! I suggest a 20% reduction in order to take into account the problem of access to markets. That coincides with the Addis Ababa Contingency Plan; it gives you about 25,000 beneficiaries. In my view, that is a very honest figure. You know that in Kobo and other places beneficiary totals are as low as 5,000, 4,000 or 3,000, so even 25,000 is more than enough, it’s reasonable.” The motion was carried: as it was an ‘Addis Ababa decision’, everyone gave in.

2. NEGOTIATIONS WITH THE ZONE

The June 2003 belg evaluations conducted in Meket woreda illustrate the flexibility of the figures and the way in which they were transformed during negotiations at various administrative levels.

At first, the woreda committee estimated that 5,624 people were in need of assistance; the evaluation team suggested a figure of 3,000. The main disagreement concerned agricultural loss percentages, with the committee claiming that 1,312 hectares had suffered 100% losses, the equivalent of 11,814 quintals. Ahmed (WFP) justified his calculations on the basis of yield per hectare, and would not accept that losses amounted to 100%: “It’s impossible! Unless there is a disaster, a 100% loss is, in my experience, impossible!” The woreda representatives explained that their estimate was based on the accumulated losses for the entire woreda, with locally lower percentages. But they maintained that seven kebeles had sustained 60% losses, while another three had suffered 100% losses. The discussion was cut short and Ahmed, who had exhausted his arguments and had no further means with which to negotiate or apply pressure, suspended the meeting without reaching an agreement, although he was furious and still disputed the 100% loss figure. His only recourse was to back up his claims by field observation and then try to convince the zone authorities.

Later that day, the team inspected the crops of two relevant kebeles from the road. Before beginning negotiations with the zone, Ahmed suggested that his team should reduce the beneficiary total from 5,624 to 4,683 - he estimated that 25% of food needs were covered by non-agricultural incomes, an acceptable percentage when compared with the previous year’s figures.

Then, Ahmed began negotiations with the zone committee by describing the good condition of the crops inspected from the road, and suggested restricting the number of beneficiaries to 3,000. The zone leader thought that a good idea, but asked how the calculation had been arrived at. Ahmed mentioned the comparison with the long term (1998-2002) figures and those of the previous year. Bekele backed him up: “Kebele 27 is exceptionally good this year.” The zone’s DPPC representative thought otherwise: “The situation in Meket is bad, and there are no possible economic alternatives. If the woreda people were here, they could defend their situation better than I can.” Zelalem added that the peasants who had suffered poor belg conditions had already begun planting for the meher, while Ahmed asserted that “the rainfall and crops confirm our findings and enable us to predict a good harvest.” The two main arguments - the percentages of other incomes and the melg/meher ratios - were not enough to persuade the zone committee. The zone leader resolved the situation by suggesting 4,000 beneficiaries, a figure that everyone accepted with relief.

F. NEGOTIATING WITH MEANINGLESS FIGURES

1. THE FRAGILITY OF LOCAL DATA

The bargaining process seems almost surreal given what it was based on - crude field estimates compiled in the form of a report (see above). In November 2003, I had the opportunity to accompany an MOA development agent as he estimated crop losses in a kebele in the Dehanna woreda. Called in by villagers to observe the damage caused by hail storms, he made no attempt to take proper measurements of the damaged areas. Roughly gauging the percentage of damage to each parcel of land, he totalled them up and used the global percentage to calculate the total area of damaged parcels. Moreover, his calculation was based on an error-prone automatic conversion of traditional units of measure (Temad or Tinde) to hectares.

From what they said, the agents did not systematically go on-site to verify the damage sustained by crops. The reports they wrote for the woreda rested on information (the number of fields and families affected) sent in by village leaders; this was subsequently transformed into standardised data calculated from land averages per family and temad/hectare conversion tables. Woreda committee evaluations consisted of no more than quick visits to estimate crop yields. No attempt was made to take proper physical measurements; a cursory inspection of the crops was considered sufficient. The woreda agents were chiefly concerned with compiling the data sent in by development agents. The precision of the figures used during the negotiations were in complete contrast to the gross approximations of the data collection methods.

The evaluation team experts were well aware of the limits of the data at their disposal. Zelalem (DPPC) told me that “The percentages of losses and land areas affected are estimated visually by development agents. In theory, there is the ‘cutting sampling methodology’, which should involve taking several 0.5 x 0.5 metre samples from each field. You remove the ears and count the grains, and these samples, taken from a specified area, give you an estimate of the yield. But this method is too difficult; there is no time and no appropriate tools. It would take an agent three months. So he comes up with estimates, which are then compiled at woreda level. There is no alternative. As for woreda evaluation practices, I know all about that: they make quick visits but rely more on information collected by agents, and on compiling area and production data, which they then turn into adjusted percentages!”

Molla, a member of the NGO Concern, made similar comments: “We either accept the data from the Ministry of Agriculture or go out and measure the fields ourselves. We settle for the calculation of the total of all the areas affected. The development agents provide estimates, suppositions. They are supposed to measure the most severely affected fields and strips, as well as the healthiest, but they don’t have the means and refuse to do it. So their estimate of crop conditions is visual. And the data expressed in traditional units of area measurement is also relative.”

Remarks by Ahmed (WFP) also evoked the frustration over the lack of accurate data: “You get agents who simply ask village leaders to provide data, and don’t check any of it. Moreover, they are not supervised, maybe two or three times a year by the woreda. As they are educated people, they don’t like spending time in peasant villages, so keep visits to a minimum. And even when an agent condescends to ask a peasant how much he has planted, he’ll never receive a proper answer! The peasant won’t even tell his own children how much he has harvested. He’s too afraid of paying the government even more tax! So if he has obtained six quintals, he’ll say it’s only four or five. All our results are subjective. And even if official data is calculated correctly, we don’t know how it is produced, if the agent worked conscientiously or even if there were any agents in the kebeles! And of course we may also calculate figures and percentages pertaining to places where the agents are unscrupulous or where there is no agent at all.” His view of the woredas was even more cynical: “The woredas adjust the deficit and production results for their zone as they see fit. They simply note tendencies and compare them with the data from the previous year. Moreover, woreda expert teams change all the time, there’s a very high turnover. As a result, there is conflict between the newcomers and the woreda administrator. The experts are usually afraid to confront the administration, so they are willing to accept the raw data provided by the on-site team.”

2. THE NEGOTIATION GAME

In a virtual dimension where meaningless figures were subjected to frenzied haggling, it seemed the participants did not seriously believe they were there to arrive at the ‘true figure’; the object was to gain the edge over one’s adversary. Negotiations sometimes included frenetic sessions with a calculator, with everyone adjusting his reckoning in order to downplay the worth of the other proposals. The intensity of these moments was in marked contrast to the inertia that characterised the rest of the meeting. As the protagonists pitted one methodology against another, drew an ace from the welter of data, contradicted their own arguments and attempted to recover their losses with a new calculation, the process began to resemble a collective attack of gambling fever.

These statistical tournaments led to some fierce jousting. A player would savour a brief moment of victory, a point won thanks to the strategic use of an argument, before re-entering the fray when another demonstration flopped - a point lost. As Ahmed remarked, “I had worked as a government agronomist for several years, I knew how data could be fabricated and figures turned into gambling chips. That’s why I look for the weak points in their logic and reports when I attend meetings. I try to demolish their arguments. That puts me in a position to negotiate and impose my point of view!” When experts - most of whom were highly qualified - were reduced to negotiators, a certain feeling of bitterness was inevitable. As Ahmed sadly noted, winning the negotiation game offered little consolation: “I’m losing my technical knowledge and skills. All I’m learning is how to negotiate with the authorities and the woreda people.”

The evaluation process was a long haul: every ‘victory’ enabled the expert to strengthen his argument in future dealings with the woredas, but every ‘defeat’ meant that negotiations would have to continue at a higher level, the zone, where all the woreda results were validated. For the evaluation teams, the zone represented the final opportunity to argue their case. As Zelalem observed, “If we can’t persuade the zone to accept our results, the region won’t be prepared to go against the decisions taken by the woreda and the zone.”

3. MAINTAINING CORDIAL RELATIONS

At woreda and zone level, negotiations concluded in a delicate balance designed to maintain diplomatic relations between experts and local authorities and to produce results which conformed to the scenarios and expected averages. The fact that the team and the local authorities remained on good terms is quite revealing. Discussions structured around technical arguments always enabled the participants to reach agreement on a narrow range of figures, close to an average of the competing claims. Although these averages relied on abstract statistics, they were finally accepted by all parties. Consensus ensured that the cordial relations essential to the continuity of future discussions were maintained.

Indeed, once a multi-agency evaluation had been completed, DPPC and WFP agents and local authority representatives would still hold regular meetings to exchange the information on which the day-to-day functioning of the early warning system depended. During the somewhat disastrous 2004 evaluation, Gebre Mayam (WFP) told me that delicacy was required, for conflict could not be allowed to jeopardise the future meetings essential to the routine collection of information: “We are restricted in negotiations, we can’t afford a split. Our WFP agents regularly visit the area and it would create problems for the future.” The rare cases of hostile relations I observed in 2002 and 2003 were referred to the regional authorities, who acted as mediators between the parties in conflict.

G. THE BREAKDOWN AND REPAIR OF RELATIONS

When the woredas sought to defend their estimates by appealing to the regional authorities, the balance of power tended to shift in the authorities’ favour: the experts’ power to negotiate on a technical front was eclipsed by political arguments. In November 2002, the teams were forced to modify their results on their return to the regional capital. The dispute concerned the woredas of Meket and Dewa Cheffa.

1. THE CASE OF MEKET WOREDA

During the first stage of negotiations, the multi-agency evaluation team and the Meket woreda committee had agreed, after the usual haggling over the percentage of non-agricultural activities, to reduce the initial 53,000 beneficiary total to 19,500. The participants shook hands and left the room with strained smiles. Two weeks later, as the team was writing its evaluation reports, the Meket authorities sent a fax to the regional DPPC, arguing against the reductions the team had imposed. The woreda leader presented the detailed calculations made by Getu at one particular meeting, but inverted the order of the calculations to produce a figure of 50,000 people, and therefore much closer to the initial proposition. At first the fax had no impact on the team’s decision; the group believed that the woreda experts had understood nothing and had juggled the figures. Moreover, Getu (DPPC) had seen the problem coming and, with the team’s agreement, had done some ‘juggling’ himself, increasing the number of beneficiaries from 19,500 to 30,100 on the basis of beneficiary averages over the previous ten years.

On the last day of report writing, as we worked in Judith’s office at the DPPC, Amara, the head of the regional early warning department, joined the group, told us about the faxed complaint and asked us to increase the Meket beneficiary total. Anteneh categorically refused: “Why should we always agree to increase the figures? We think the figures should be reduced. What is the point of our work?” Amara said that had nothing to do with it: according to other reliable sources, there had been recent localised frost damage in certain villages. He mentioned teams from the NGO SOS Sahel “which had inspected and had even probed the crops that had been affected.” Getu was not convinced: “But they didn’t mention it when we spoke to them.” Judith supported him: “The zone itself has not said anything.” Anteneh pointed out that the group had seen nothing during the field trip.”

Ignoring the avalanche of protest, Amara notched up the pressure by calling the team’s competence into question: “If there is a problem, we are going to be confronted with malnutrition ... we will have to make use of a team at federal level!” Anteneh offered a final argument: “Go ahead. How can we change the figures if only one parcel of land has been affected? That’s not enough.” Amara brought out the heavy artillery: “OK, there is no more time to discuss this today. We have to finalise the data at regional level. We will have to compromise and increase the number because of the frost. There has been a misunderstanding and an inadequate exchange of information between you and the authorities. We must come to an agreement, which takes into account the fears expressed by the region. If we can’t make up our minds now, you’ll have to meet the Commissioner!” Getu capitulated: “OK, so there was frost, let’s take it into account.” Fissaye then suggested adjusting the figure according to the ten-year average, which would leave the door open for a compromise. The total was increased to 38,000 people. Case closed.

2. THE CASE OF DEWA CHEFFA WOREDA

The Dewa Cheffa woreda authorities had also faxed a complaint to the region, stating that the evaluation team had accepted only 30,000 of the 60,000 beneficiaries counted locally. Claiming that the production deficit figure advanced at the time of the visit had underestimated the total by almost 100,000 quintals (!), they suggested that the beneficiary total should be increased to 80,000. Anteneh, Getu and Judith re-did their calculations and, after re-estimating the proportion of non-agricultural income in household food budgets at 20%, increased the beneficiary figure to 38,000. All the team members were in agreement with the exception of Fissaye, who believed we were too flexible and should have taken a firmer stand!

While the Meket negotiations were under way, Amara, the head of the regional early warning department, had tried to increase the Dewa Cheffa beneficiary figures by advancing two arguments: the averages for the previous two years and the recent FAO report on Dewa Cheffa, which depicted the situation as critical. He kept repeating that historically, the district was particularly prone to recurrent droughts. As we were all exhausted by the Meket negotiations, he decided to leave it at that, telling us we would be meeting the Commissioner that evening and making it clear that “in any case, we would have to find a compromise.”

At eight p.m., we found ourselves in the office of Aderaw, the regional DPPC Commissioner. He talked about the very tight deadlines imposed on us and said that the donors in Addis Ababa were applying considerable pressure because they wanted the data before Christmas. Given the “crisis”, the donors wanted to move quickly. Before the haggling commenced, Anteneh pointed out that challenging our Dewa Cheffa results amounted to disputing the entire output of a team of experts. Aderaw adopted a conciliatory tone, congratulating us on the intensity and high quality of our work. He accepted the readjusted figures for the Meket district. Everyone agreed on the long-term aid average. On the other hand, there was a shortage of information concerning Dewa Cheffa, an error for which we were not responsible: “You don’t have all the information you need on which to base a decision. The collection and analysis of data by the district and zone authorities was flawed. You can’t base your decisions on their data. There are some very alarming FAO reports. There is a risk of failing to respond to an emergency. In order to compensate for their error, the target should be at least 45,000 people, not 38,000. And even at federal level, they are not going to accept the 45,000 figure.” Anteneh compared the 2002 agricultural deficit and beneficiary figures with those of 2001 and stood firm: “We can’t justify these figures to the donors!”

The discussion grew heated, and Aderaw furiously criticised the inability of the local authorities to come up with valid figures. Adopting a more magnanimous tone, he added: “It’s obviously not for us to punish them for their behaviour.” Judith insisted that the local authorities’ error-ridden estimates were not enough to alter the figures we had established. But Aderaw was adamant and kept referring to a possible disaster: “If we are mistaken, we risk failing to respond to an emergency.” Anteneh capitulated: “We are not here to create problems.” All that remained was to agree on a figure. Aderaw returned to the 45,000 figure and, exhausted, we accepted it.

H. THE MARGINALISATION OF EXPERTS

1. INCREASING POLITICAL PRESSURE

In November 2002, the evaluations were subjected to heavy pressure from the government, which wanted a national total of 15 million beneficiaries. Between October and the publication of the final round of evaluation results at the beginning of December, the government adhered to the worst-case scenario. By mid-November, the ‘worst case’ figure, rounded up to 15 million, had already been announced in the broadcast and print media by representatives of the WFP, the DPPC and the Ethiopian Prime Minister. Institutions were predicting a national disaster even before the conclusions of the November evaluations had appeared.

On 13 November, the government newspaper Ethiopian Herald ran a headline claiming “Millions threatened by drought and food shortages in Ethiopia”. The WFP announced that according to the worst-case scenario, 14 million people would be affected. The articles in which the Prime Minister, Meles Zenawi, expressed his own views no longer referred to scenarios, but to famine and the possibility of 15 million victims! On 12 and 16 November 2002, The Ethiopian Herald published articles claiming that “Ethiopia is facing a more severe famine than that of 1984, which claimed almost a million lives ... The number of people suffering from starvation might reach 15 million by the beginning of next year if international donors do not support an international aid effort... If the 1984 famine was a nightmare, this one will be truly appalling. Meles predicts that the number of drought victims will be three times greater than in the previous famine.” Note the seamless transition from a fairly objective tone to one of drama and emotion - the comparison with 1984 - complete with images.

At precisely the same period (mid-November), the evaluation teams were still writing their final reports. Taken together, the total number of beneficiaries nationwide hovered around the ten million mark. But the ground had been prepared to facilitate its gradual inflation to 15 million. The evaluation teams thus arrived at a figure of 11.3 affected persons. This was revised to 12.6 million in April 2003. Two years later, Mohammed, an expert with the regional DPPC, told me that his team’s figures for the North Shoa zone evaluation in November 2002 were upwardly revised (without negotiation) by including the belg-dependent populations that were likely to be affected ... in July 2003! The overall results appear to have been artificially inflated by adding belg-dependent populations.

For these reasons, the belg harvest evaluation conducted by our team in June 2003 resulted in a prime example of a sudden turn of events by the region. The total beneficiary figures for meher- and belg-dependent populations were systematically reduced from the padded totals established in November 2002. The beneficiary total emerging from the July 2003 belg evaluations finally levelled out at 13.2 million people.

2. DECREASING POLITICAL PRESSURE

In November 2003, political pressure sought to decrease figures. The new ‘Safety Net’ policy adopted by the government and donors was expected to lead to a marked reduction in the number of people dependent on food aid.

When the multi-agency evaluation teams began their final round in late 2003, woreda estimates had already been downwardly revised by the local authorities. It was true that the 2003 harvests were much better than those of the previous year. But the cuts at local level were considerable, and exceeded by far the traditional scenario-based adjustments. The teams were unable to negotiate upward revisions. As a result, the final Amhara region estimates indicated that no more than 450,000 people required food aid, as opposed to an average of two million during previous years. It was clear to all the experts involved that “Addis Ababa had issued orders” to the woreda authorities, which had obeyed them to the letter and had reduced the figures.

The initial multi-agency evaluation results therefore revealed a drastic fall in regional beneficiary numbers. The estimates were so low that the DPPC in Addis Ababa issued a counter-order telling the teams to distinguish between populations suffering from ‘chronic’ and ‘acute’ deficits, in the hope that those experiencing a chronic food deficit had been included in the breakdown. Zelalem got down to the task and consulted his archives and statistics. But there was clearly no time to conclude the exercise before the official handing-over ceremony in Addis Ababa.

During the hand-over, the impossibility of distinguishing between the two population types and thus justifying the national total, which had also been reduced by one million beneficiaries, influenced the official publication of the final figure. Tashome, the head of the Addis Ababa early warning department, suggested that the evaluation timetable began too early: crops that had not yet ripened could not be estimated accurately. He closed the public meeting by saying, “The date set for the evaluation was too early; preparation at woreda level was inadequate, which affects the quality of the evaluation results. The post-harvest evaluation will give us a clear and conclusive picture for 2004. However, it seems that production is better than it was last year. We even expect some surpluses, as in the Omoro region. But there are still pockets which need assistance. The problem is trying to distinguish between ‘severe’ and ‘chronic’ in each area, for the two have been mixed together. Unfortunately, extra evaluations are needed to cover chronic needs. The new food security framework should provide some guidance.”

The figures were published two weeks later, on the date foreseen. They predicted two million beneficiaries for the Amhara region and 7.2 million at national level. According to Mohammed of the regional DPPC, the Amhara region calculations were the work of Zelalem, who had based his findings on beneficiary averages.

Several factors have been advanced to explain the manipulation of the figures. First, the order to cut beneficiary numbers was alleged to have gone directly to zone and woreda administrators, thus bypassing regional and central DPPC offices. Second, signs of panic among the donors might have influenced the upward revision of totals. Finally, the revised figures erased the work carried out by the evaluation teams. Nevertheless, the official report - Humanitarian Appeal for Ethiopia - stipulated that the results were based on “a thorough evaluation of the food situation, etc.” and claimed that “multi-agency evaluations predicted agricultural production comparable to the last five years in the high plateaux zones.”

3. THE REVERSAL OF ETHIOPIA’S FOOD AID POLICY

In November 2004, the formal adoption of the ‘Safety Net Programme’ altered the approach to harvest evaluation. The programme was supposed to introduce a new type of aid for populations whose food deficits were linked to a progressive decline in their living conditions. Financial and/or food support would henceforth be conditional on participation in work for the collective good; there would also be the possibility of investment credits in the form of heads of cattle, hives, etc. The goal was to extract the programme’s beneficiaries from food insecurity in three to five years and enable them to survive without government aid. Populations affected by an acute and localised natural crisis would continue to receive traditional emergency aid.

Although the policy was presented as a turning point in the history of aid management, nothing really changed at basic level. The peasants continued to receive food aid and were subjected to work for collective programmes. But aid was henceforth regarded as a transitional measure; the ideology behind the policy hinged on voluntarism and the encouragement of individual responsibility, and denounced the ‘assistance machine’ and ‘dependency’ as the negative consequences of free food distribution.

By the autumn of 2004, the programme’s implementation was accompanied by intensive information campaigns run by administrators at every level of the hierarchy. Cadres were ordered to organise village meetings and inform the peasants of the new developments. The walls of every administrative office were adorned with posters castigating the resort to aid, which was likened to a form of begging and condemned as shameful; work was glorified as a salutary value. These slogans became absorbed into the civil servant’s stereotyped discourse, whose practitioners denounced the “dependency syndrome” and the idleness of the peasantry.

The policy was accompanied by the reinforcement of the resettlement programme that had begun at the beginning of 2003 (see page 27). The programme seemed doomed to failure in the Amhara region, where the peasants refused to settle in the Metema camps set up in Gondar zone. Of the small number of pioneers who moved to the site in 2003, practically all had returned, chiefly because of the poor conditions and risk of malaria. According to the official version, villagisation was a purely voluntary affair - peasants were not compelled to migrate. But in practice, those who refused to leave could no longer count on receiving external assistance. ‘Free’ food aid in the peasant’s home region was henceforth perceived as an obstacle to their displacement and as an impediment to their development through labour.

In 2004, aid to the populations of villages in Wag Hemra and North Wollo was restricted to those who were able to participate in collective “development” work. The elderly, the disabled, pregnant women and children from large families were excluded. The kebele, woreda and zone authorities I spoke to justified this decision - and brooked no discussion - by evoking the fight against the “dependency syndrome” and the negative effects of free aid. However, by January 2003 it was being made clear at formal village meetings that food aid was available in the resettlement camps. For populations whose harvests provided enough food for half the year, the abrupt termination of aid was synonymous with forced resettlement. At local level, food aid served as both carrot and stick in the furtherance of government programmes.

In the light of this context, the evaluation teams examining the 2004 Meher harvests were tasked with quantifying populations suffering from ‘acute’ shortages which would make them eligible for emergency aid under the usual conditions. From the outset, the new categories were defined by Addis Ababa, which distinguished populations with no food needs from those receiving provisional ‘Safety Net’ aid and those receiving emergency aid. The ‘Safety Net’ population of each woreda, zone, etc., was calculated on the basis of the food aid beneficiary average over the previous ten years. The 2005 total rose to five million persons. In the field, evaluation procedures and methodologies remained unchanged. The teams were not given the opportunity to discuss the criteria for each category, although they believed the distinctions were ambiguous and highly theoretical. As they travelled from woreda to woreda, they encountered marked variations in the inclusion criteria. The number of ‘acute’ beneficiaries was calculated by means of a methodological combination comparable to that used in previous years. In 2005, the national emergency aid beneficiary total eventually reached 3.2 million.

In November 2004, the multi-agency evaluation period coincided with an intensive propaganda campaign mounted by kebele and woreda administrators. The evaluation teams, already pushed for time, were also deprived of consolidated data to use in the negotiations. They were almost forced to plead for meetings to be arranged so that they could finalise their work. The tendency for beneficiary numbers to decrease became more pronounced, and some woredas and zones even declared “zero beneficiaries”. The case of Wag Hamra zone provides a good example. Preoccupied with organising village information meetings, the Dehanna woreda administrator initially refused to talk to the team, which then went directly to the zone leader, with whom it negotiated the possibility of collecting technical information as long as it had nothing to do with beneficiary totals! The team eventually left the zone without having negotiated a figure.

Over the next few days, Mohammed (regional DPPC) came up with the idea of a beneficiary total based on the figures for North Wollo. Mohammed was then persuaded that the decision taken by the head of Wag Hemra zone was due to a misunderstanding of the new policy. He telephoned his regional superior and asked him to speak to the head of Wag Hemra so that a beneficiary total could be finalised, but his efforts were fruitless. I later learned that the Wag Hemra chief later contacted a minister and asked him to ensure that the federal DPPC ignored any requests from the expert teams. The same procedure was probably applied by the authorities of Kalu woreda when they reduced the 20,000 total to 4,000. In this instance, the marginalisation of the expert was so extreme that his ability to influence the allocation process was reduced to zero.

4. EXPERTS KNOW THEIR LIMITS

The experts were fully aware of the limits of their technical expertise when final decisions were shaped. Mohammed (DPPC) told me that his role was simply that of an adviser to politicians and bureau chiefs. “As experts, we strive to approach the reality of a situation as best we can. But the politicians try to revise or increase beneficiary numbers after the evaluations, as for the 2003 Meher, when we were forced to add belg-dependent populations. To politicians, we are simply advisers. We provide the data. They take the decisions, not us. We are always expected to increase or reduce the figures after the evaluations. Four or five years ago, they had to come down. Two years ago, they had to rise. This year, they have to come down again.”

In the expert’s view, any attempt to exceed these prerogatives entailed the risk of some form of coercion. Mohammed felt that his job was at risk: “We have no choice - if we go against political decisions, we can be sacked.” For others, the consequences ranged from censure to prison. In November 2003, for example, the entire team expressed their criticism of the political manipulation that led to the reduction in beneficiary totals. When the members sat down to write the reports, they could not justify the cuts imposed by certain woredas. Befekadu (CIDA) referred to the political choices made by the woreda committees. I suggested that he should mention this in the report. Girum, the young team leader, was horrified: “If you write that, you’ll end up in prison!” Jokingly, I replied that as a food security agent, he could then bring me some oranges. He replied: “I couldn’t even do that, because we’d all go to prison.” Zelalem, the DPPC representative, took no part in this exchange. Befekadu later attempted to smooth things over: “Girum is young and tends to panic! If we wrote such things, we’d never end up in prison. But explanation like that would never be accepted in a report, the federal or regional DPPC would automatically remove it. You can’t put that in a government report.”

The view of WFP agents like Ahmed was hardly more optimistic: “Working with the WFP does not mean I can change anything. We are simply messengers. We deliver the authorities’ reports to the Addis office, or receive a note when the food arrives: ‘The food is arriving, be ready to receive it!’ That’s all! The authorities toe the official line; there is no flexibility.”

5. MESSENGERS OF POWER

It is clear from this ethnographic reconstruction that the experts’ room for manoeuvre when defending a technical approach was extremely limited. The exercise of their expertise was restricted by politics. When they began their work, they were already handed a framework - the predetermined sets of figures which would influence the outcomes. At the evaluation’s conclusion, any deviation from the framework was corrected by appealing to higher authorities, thus reversing the balance of power and bypassing the experts’ decisions. And ultimately, when priority was given to politics, the expert was definitively excluded from the decision-making sphere.

On the limited occasions when they were able to work without political constraints, the experts found their power neutralised by the absence of any reliable methodology. They had to resort to the empirical approach, observing the agricultural landscape and comparing it with previous years and other districts. This work was notable for its originality and pragmatism, the visual assessment of crops was adapted to the constraints of the evaluation and the absence of more sophisticated tools. The use of these ‘subjective’ methodologies evolved as the evaluation progressed, but it was a matter of proceeding by trial and error, and making deduction after deduction. Evaluation exercises were therefore reduced to haggling over the results with local authorities. Haggling was a race against time: the aim was to find the best match for a predefined scenario, and the arguments rested on the combination of methodologies and an opportunistic selection from a mishmash of data. The quality of the negotiations was itself inevitably linked to the composition and dynamic of the evaluation teams and local committees.

The inability of the teams to arrive at an objective estimate or to counter political pressure meant that they were reduced to passing figures along an information chain that stretched from the villages to Addis Ababa. Although they had to work with data which had originated from crude estimates which had already been subjected to considerable revision, the teams provided the ‘professional stamp’ that enabled international decision-makers to accept the figures. These figures maintained the illusion that evaluation was a neutral process. Despite themselves, the teams whitewashed the data so that it validated every political decision concerning the management, increase and reduction of aid.

Negotiations between local committees and evaluation teams were conducted in mutual blindness; both parties used figures that bore no relation to the reality of the situation. Numbers were constantly re-used again and again by referring to the averages for previous years (which had resulted from similar practices) and by ‘cutting and pasting’ during the report-writing sessions. Devoid of its primary meaning, the evaluation’s true significance emerged when the negotiation game commenced. Its function was to facilitate the construction of a diplomatic consensus between the actors at the various levels of the administrative hierarchy, governmental actors and donor agencies. The way in which the EWS operated was in keeping with the guiding policy described by sociologists of organisations: it sustained the equilibrium of the institutional system for managing food insecurity.

The evaluations took on an air of absurdity which was accentuated by the protagonists’ awareness of the limits of their data and methodologies, and by the fact that highly qualified experts had to resort to the debilitating business of haggling. Some experts loathed this task: “This is a really crap job! I’ve had it up to here with working as if I was buying a goat in a market.”

The experts’ awareness of the risks of censure and of losing their jobs leads me to suggest that they were subjected to real coercion by an authoritarian administrative machine. In this instance, it was expressed by the dictatorship of figures, which was itself generated by donors’ expectations, and by the rigid format (covering both form and content) of the evaluation reports.

Finally, the experts’ distance from the world inhabited by the peasantry ensured that they had no grasp of the realities of village life, and an even looser comprehension of the modalities of aid allocation. Even had they wished, they were incapable of influencing the political manipulation of aid that was likely to occur at village level. Whether assumed or genuine, their lack of awareness enabled the manipulation of aid to continue without protest. It also maintained an information and decision-making system that was divorced from reality, which explains the omissions of pockets of malnutrition, such as those which appeared in the spring of 2003.