On the Space Between Failure and Success

Science Stories: The Art of Scientific Storytelling

The City of Knowledge

On the carved banks of the Panama Canal lies the Ciudad de Saber. This City of Knowledge keeps watch over Miraflores Locks, one of three ladder steps engineered into the canal, raising or lowering freighters 54 feet on their voyage between the Atlantic and Pacific. The City of Knowledge and its learned occupants enjoy this privileged real estate because it was once Fort Clayton, the United States Army South Headquarters. Fort Clayton was charged with vigilantly protecting and patrolling this 82-kilometer aqueous artery until President Clinton turned over the Canal to the Panamanians in 1999. What do you do with the fort of a country who dug a canal through your jungles and then moved in for nearly a century? Well the Panamanian presidents Nicolás Ardito Barletta and Ernesto Pérez Balladares decided to symbolically transform the hodge-podge of pastel stucco buildings and their red tile roofs into a center for knowledge exchange. Thus, when Bill Clinton handed over the keys to Presidenta Mireya Moscoso, the experts moved in.

I arrived in the City of Knowledge much later. It was June and the mango trees strewn across the campus were heavy with fruit that would fall at the suggestion of a breeze. Once the fruits were on the ground, they would be consumed by flamboyantly colored insect foragers. I know this because I spent my early, slow days in the City of Knowledge staring out my window watching the quick drop and slow decay of the yellowing red fruit. I arrived here as an intern assigned to the climate change adaptation team of the United Nations Development Programme (UNDP). UNDP is one of many United Nations agencies that pays rent and exchanges knowledge in this lauded community. Panama’s highspeed internet, political stability, and easy access to foreign goods (thanks Panama Canal) make it a great home base for regional experts.

On my first day, I was baptized into the faith of expertise. I was anointed with a freshly printed photo ID strung declaratively on a blue logoed lanyard, the security coordinator placing it solemnly around my neck. I got assigned a desk between a high-powered program assistant and a natural disaster response advisor. Sitting at my desk, languidly doing the 16th module of a 32-module online course on UN safety norms and procedures, I started to watch the mangos fall.

As a second-year Ph.D. student, I came to the City of Knowledge on a pilgrimage, seeking a worthy topic for my dissertation research. Scratch that. A meaningful dissertation topic, one with impact and relevance. Okay, let’s be honest: I had gotten a little desperate; I was looking for a dissertation topic. Period.

The stakes for adaptation projects are high. No one wants a failure and, therefore, it is hard to find one among the project evaluations.

Success and Other Stories

When I arrived at UNDP, I was an aspirant scholar of how projects can help prepare communities to respond to the impacts of climate change. Emphasis on aspirant. UNDP implements over half of all internationally-funded climate change adaptation projects in Latin America and the Caribbean. So, you can imagine that getting this internship felt like divine providence to a doctoral student without a research project. Emphasis on divine. My mission as the department’s sole intern was simple. I was to write a publication about UNDP’s most successful adaptation projects in the region. Emphasis on successful. UNDP’s African and Asian regional hubs in Addis Adabi and Bangkok had already published a glossy report featuring their adaptation prowess, demonstrating their slam dunk strategies and funding worthiness. It was time for the Latin America and Caribbean team to establish themselves as the undisputed, preeminent regional experts on climate change adaptation. Emphasis on the. And I was the unpaid intern to do it. Grudging personal emphasis on unpaid.

With my mission in hand, I began by combing through the internal document server looking for successful adaptation project evaluations as the mangos fell. A project that installed early warning systems in Colombia now alerts 12,525 people when a landslide or flood is happening. That evaluation was definitely positive. Or perhaps the project that worked with the Cuban government to create 15 new marine conservation areas, buffering the ravages of hurricanes? Also labeled successful. Huh. Climate proofing water and agricultural systems in Haiti? That also was stamped successful. As I reviewed evaluation after evaluation, I began to realize that all the evaluations for all the projects carried the lofty title of “successful.”

But can all these projects really be successful? Sure, sometimes an evaluation would mention a pesky issue with staff turnover or a near-catastrophic delay in project progress, but these storm clouds were promptly dispersed in the following paragraph with incoming staff arriving in chariots at the last minute to save the day and early delays metamorphosing into unstoppable momentum towards sure triumph just in the nick of time. As a note on my earlier sentence, evaluations are very dry and matter of fact, so the heroic gestures and chariots are my efforts to spare you from evaluative prose.

Perhaps the question is what success even means. In bigger and smaller, permanent or more fleeting tastes, we have all enjoyed success. It is sweet. That gleaming medal pinned to chest. The standing ovation after the stuck landing. The music swelling and the skeptics admitting they were wrong. The rave reviews. The bloated bank account. Success might mean different things to each of us, but it always tastes good. It brings vindication. It brings recognition. It brings options. And in the case of adaptation projects, success means future projects and crucial funding.

For its thematic expertise and administrative prowess, UNDP captures around 8 percent of the budget of every adaptation project it implements. And before we label them charlatans it is important to note two things: 1) all the international agencies who implement adaptation projects charge a fee for their supervision and know-how, and 2) donor nations like my own have continually cut their donations to the UN system and are partially responsible for this emerging business model. So, it turns out there are high stakes for UNDP and every other adaptation project implementor (e.g., the World Bank, Food and Agriculture Organization, Inter-American Development Bank, and so on).

Also, the climate funds that finance projects want only successful projects in their portfolio. After all,  just like you stop paying the plumber that fails to unclog the toilet, the nations that put their tax dollars in the Adaptation Fund or Green Climate Fund are weary to write another check if these funds finance failing projects. The stakes for adaptation projects are high. No one wants a failure and, therefore, it is hard to find one among the project evaluations.

“Plop!” Another mango down. I look to see where it lands, deviously hoping it splattered on a parked car, and have to hastily remind myself to focus.

“Disculpa. So why was this particular project more successful than others you have implemented?” I have decided to put aside the server and start talking to the people in Panama and across the region who created, ran, and evaluated adaptation projects. Skype call after Skype call, I ask about what made a project a success, a failure, and the stuff in-between. The stories I hear are rich and complicated. There was the project that stalled for three years and then everything happened in a frenzied three months. In another project, the government stakeholders didn’t get along and blocked any intervention. Sometimes communities rejected the project and new communities were found—or not.

What rang true in every conversation I had was that these adaptation professionals cared deeply about their work and that these types of initiatives are hard. These projects are hard for lots of reasons. Many of the project measures haven’t been implemented before, and if they have been, not in the context of looming, capricious climate change. Also, there are behemoth bureaucracies at every level of these projects that unpredictably stymy and smooth processes from project design to close. Lastly, you need to involve a lot of people in these projects, necessitating the obvious qualification that involving a lot of people almost always throws sand in the gears.   

I also heard stories of success far outside the cookie-cutter evaluation metrics. Over beer, a regional adaptation expert recounted his enduring friendship with local project partners as personally meaningful. A Colombian adaptation specialist recounted how he carried his hard-fought lessons from one adaptation initiative to the next. One adaptation project consultant described how she learned water saving techniques from local farmers and integrated them into workshops across the region. Through these conversations, I came to realize that success is not an up/down vote by project evaluators. Nor is success confined to the formal project evaluation metrics, but rather is often eclipsed by less orderly types of success at different scales and for different actors.

As an American, I was raised on a steady diet of success mythology. I expect movies to end with a swelling score and a satisfying resolution where our beleaguered protagonist reigns victorious. I have drunk the delectable Kool-Aid of optimism, believing every setback is merely preparation for my own victory. I have read Stephen Covey’s classic The 7 Habits of Highly Effective People. I was an all-state athlete. I got first prize in my sixth-grade science fair. I am a Fulbright/Luce/PEO/National Science Foundation Graduate Research/Carson/Emerson National Hunger Fellow. In 2010, I won $50 from a scratch-off ticket and I still talk about it. Success matters to me and I work hard.

Failures, on the other hand, are intimate and unseemly. They tend to blister when exposed to light. But our failures are also often bitter bellwethers and a needed rap across the knuckles. I should know. In a final bid to save a withering six-year relationship, I pleaded with my partner in the final moments before his flight to stay. He declined. I organized a student photography exhibit at a prominent cultural center that no one ever came to see (not even the artists). I shattered my left hand when I fell off an Andean climbing route and it is now irrevocably misshapen, serving as a testament to my recklessness. I spent most of 2017 fundraising for a research project that didn’t work. Just this past month, I received three emails from national writing fellowships, informing me that my dissertation wasn’t worthy of funding. So, while I may have been raised on a diet of success, failure is often on the menu.

With this in mind, I get why climate change evaluators, implementors, and advisors are loathed to graphically depict projects’ unbecoming failings and unredeemable faults. How embarrassing! How bad for business! How unexpert of them! This is the City of Knowledge after all. But my empathy is limited, because while the stakes for UNDP and other project implementors are high, aren’t the stakes for the communities on the frontlines of climate change even higher?

 
The Real Stakes

I did find my dissertation topic in Panama. I decided to follow one of the many adaptation projects I wrote about to Ecuador. I got curious because in all my server spelunking and conversations with experts, no one seemed to know how durable project success was. These adaptation projects tend to last five years, at which point all the government officials, our experts, and international funders move on to the next voting block and feather in their project portfolio. Perhaps it is the lack of money or our shared fear of confronting failures, but post-assessment of these projects is almost never conducted. Towards this end, I have spent much of the last two years traveling across Ecuador to visit these forgotten adaptation projects.

Like many mountain ranges of the world, the Andes guard and protect water in the form of glaciers and highland wetlands, slowly releasing its aqueous assets throughout the year for people and other organisms on the lower slopes. As the climate warms, these glaciers and highland wetlands are unable to store water as they did before, instead quickly releasing it as blue melt or vapor. As if that isn’t hard enough, the seasonal rains have also grown less reliable. Farmers are faced with increasing uncertainty about how to sow and water the fields that provide them food and income.

So, let’s be presumptuous together and think like our experts at the City of Knowledge to find a solution:

Step 1: Identify the climate change problem: less water availability. For the jargon lovers amongst us, we can call that the “hazard.”

Step 2: Determine how susceptible our communities are to that climate change problem, their “sensitivity” if you will.

Step 3: Find a solution that will decrease the sensitivity of the farmers to this climate change impact and/or something that will increase their “adaptive capacity” or ability to respond to the hazard. The solution doesn’t need to be cutting-edge or high-tech, it just needs to work and fall within the project budget. Let’s go with a classic solution: increasing water storage via reservoirs or tanks and installing irrigation networks to deliver that water to farmers in uncertain, dry times.

Step 4: Implement said solution. Next we need to work with local governments, organize farmers to form irrigation associations (or call on existing associations), get the necessary studies done to design our project, and construct.

Then the project ends, and we leave. Our job is done, but what happens next?

I am with Don Julio, the subtle and steady president of an irrigation association, who agreed to meet me on Sunday afternoon to visit their reservoir. I drive us to the base of a steep slope and park where Don Julio tells me to park. And then we start to climb. For a while I think we might climb at an even pace with the setting sun, delaying its inevitable defeat by the valley wall, but we are slower than the rotating earth. Regardless, we climb. The road is deeply rutted with chunks eroded away by heavy rains and finished off by gravity.

I ask Don Julio, “How did you get the trucks up here to build the reservoir?” He replies ominously, “It was different then.” We climb on until we finally arrive. This is one reservoir of many. It was part of the $3.5 million project that I chased from the Panama Canal to Ecuador. This irrigation association is one of eight in this small parish that participated in the project. And this parish is one of tens nationally that participated the broader initiative, which ended in 2015.

Scholars like myself often fret about the challenges of measuring success. After all, what are the right metrics or methods and over what time span and for whom? But when I crest the hill at Don Julio’s side, I don’t think anyone using even the most conciliatory metrics or methods could call this a success. Before me is an empty reservoir.

The black nonpermeable membrane is cracked into linear incisions, revealing the bone-colored earth beneath its peeling stripes. Small plants have started to grow through the cracks. The reservoir’s walls are still a perfect 60 by 40 by 20 feet rectangle and the expectant intake pipe waits. This reservoir never filled. They ran out of project money to complete the intake and rainfall was insufficient to fills its optimistic dimensions. The irrigation association now exists as a memorial to a phantom dream, hoping that the state will intercede and reactivate the project, finally delivering the water that was promised.

Conceptually, failure is an overly blunt proclamation. Just as success can mean different things, there are also many ways to fail in adaptation projects and for just as many reasons. And while the reservoir I visited irrefutably failed by even the most generous standards, most project outcomes are more complicated. Perhaps the project worked for a time, but natural disaster or human error struck, and the reservoir ran dry. Or perhaps the reservoir functions at half capacity due to management issues. Or the water only flows to the farmers who have the resources to fix pipes that have exploded due to poorly designed pressure gauges. Or the reservoir works swimmingly but there is a tear deep in the impermeable membrane that grows little by little every rainy season and will eventually need to be replaced for $20,000, a steep price tag for subsistence farmers.

The reality is that most adaptation projects I have found in my two years of research have their own complicated stories, their own histories and caveats. As I diligently document the story of each project site alongside my research team through hundreds of surveys, interviews, participatory infrastructure maps, and focused group discussions, I wonder whose sense of success matters and whose responsibility it is to tell the stories of failure and fight for success after the money has dried up alongside the reservoir.

In our society, neither scientists nor experts are supposed to be wrong or uncertain. And so, we aren’t, because we omit all the data to the contrary.

Failure and Other Realities

A Saturday storm has ushered in a pushy wind. The mangos hit the pavement of the empty parking lot, splatting then rolling into the peripheries of the asphalt. No one is at the office. An ever-toiling graduate student, I have brought my weekend work habit with me to Panama. The 60-hour work week feels less valiant when no one is there to bear witness because they’re at the beach. Regardless, I enjoy the free coffee, air conditioning, and sense of petty rebellion as I play Solange at full volume. I am leaving soon, heading back to graduate school for my final course credits and a part-time gig. I am putting the final touches on my report. It looks and sounds and feels just like the other regional reports. I have highlighted seven successful projects. I cited climate science. I meticulously recounted the accomplishments of each initiative. I included pictures and beneficiary testimonials. I vetted each case with the national and regional staff who implemented the project.

But on this weekend, since I am technically off the clock, I permit myself to wonder about the stories that didn’t make it into this report. What about all that inelegant learning that happened when things didn’t go as planned? I wonder if it would have more, less, or similarly useful to frankly talk about when these projects stumbled and when they fell. But who could even tell those stories? The consultants contracted to produce the mid-term and final evaluations aren’t going to do it. They know that the next multi-thousand-dollar evaluation contract depends on this evaluation being a positive one. And it isn’t hard to apply a rosy lens when their evaluation fact-finding mission entails only a handful of days “in-country.” They interview project staff and government officials. They might then leave the capitol for a highly curated mini tour of beneficiary communities. Those communities will have been hand-picked by project implementors. The local leaders will welcome them with glowing reviews, local fare, and a photo-op next to a gleaming reservoir. It pays to be a model community when the next round of funding comes around. But what about the communities that don’t make it onto the tour? Whose responsibility is it to tell these complicated stories?

As a scholar of success, failure, and the more probable space in between, I am aware that despite my best science communication efforts, few are listening. The international community of practitioners doesn’t want to dwell on the unpleasantness of a debunked  “theory of change.” They are too busy with the next innovative solution. Instead, those lessons are just bottled up and preciously hoarded by the few who were there when the ship ran ashore. They silence these experiences just like us scientists, who only discuss our failed experiments in the quietest of whispers in the dead of night to anonymous bartenders who keep refilling our glasses. Because the thing is, in our society neither scientists nor experts are supposed to be wrong or uncertain. And so, we aren’t, because we omit all the data to the contrary.

Sheila Jasanoff, the famed scholar of science and technology, has called for us as scientists, experts, policymakers, and knowers to employ “technologies of humility” that make explicit the ambiguities, indeterminacy, and complexity at the heart of our big endeavors and their fundamentally moral foundations. What if adaptation experts and scholars employed these provocative technologies of humility? What if all the unglamorous, sordid hard stuff was immortalized in text alongside our adaptation victories? Perhaps this report would be a bit less glossy, but I bet we’d do a better job next time.