proposal for iCampus to advance educational uses of expeditionary photography

This is a proposal for work on educational applications of networked photography to improve the ways people share and learn from pictures.

The specific driver is a unique photographic expedition to Bhutan in the first two weeks of March, 2002; and the larger context is the formation of a major new program at MIT, Global Opportunities, that will organize expeditions to spur vigorous scientific fieldwork and engage skillsets from across all departments of knowledge at MIT.

The Bhutan project requires taking 20,000 photographs that document the lives of children in one of the world's extraordinary cultures. This presses on several research problems related to digital photography (better capture techniques, including pervasive use of GPS and other metadata; better archiving, sharing, and searching techniques; and new presentation methods that tap large photobanks). And it is vital that any and all expeditions conducted by MIT or others use the very best image capture and sharing (and learning) techniques.

This proposal is for $175k of seed funding from the Microsoft iCampus program. Our work will produce both specific educational results for the development of schools in Bhutan (and other developing nations), and lay the groundwork for improved expeditionary practices at MIT and elsewhere. Partners include the Internet Archive, EPALS.COM (the world's largest internet classroom exchange), the Boston Museum of Science, and the Royal Government and Royal family of Bhutan, who have given entrée and special permission for this expedition.

Contents:
1. Background
2. Plan: an Expedition to Bhutan
3. Timetable, Budget and Delivery
4. Summary and Impact
5. Team Contacts, Partners and References


1. Background

— the importance of expeditionary work in science and science education;
— the common needs of field photography.

Gaining a berth on a great scientific expedition can be a life-altering experience for a young scientist or engineer. If the twenty-two year-old Charles Darwin hadn't lucked into a five year voyage on the H.M.S.Beagle, not only would he never had laid eyes on the otherworldly Galápagos, or been inspired to write the Origin of Species, he'd likely have continued in his modest academic rut: he'd have gone on to pursue a doctoral degree in his field — divinity! — and wound up as a country pastor. Pastor Charles Darwin, creationist.

It is important to undertake these trips. They fire up intellectual adrenaline. They help people grow. And they make intense demands on engineering technologies. How do you measure and transmit the live weather from the top of Mount Everest? Film a giant squid? Get a decent engineering handle on the degradation of coral reefs? Solving such problems taxes the limits of ingenuity. Science learns enormously from these enterprises. (From Technology Review, see: Engineering Odysseys and A Technology Corps)

Sharing the results of these field efforts is crucial, whether for niche advancements in science, or "classroom" teaching (whatever that may come to mean), or the broader education of the lay public. To this end, image sharing techniques play a critical role. Consider: if anyone has truly "taken pictures farther," it's the National Geographic Society. Yet their famous decision to include photographs in the journal was so controversial at the time that it nearly brought about a fatal schism in the organization. Nowadays, it's impossible to imagine the Geographic without imagery. Think of Hurley's photography of the ill-fated Shackleton expedition to Antarctica; Washburn's spectacular aerial mountain photography; or the stunning IMAX films, like Everest, all of which in one way or another, are gripping educational vehicles. MIT Professor Doc Edgerton invented the underwater flash, and pioneered underwater photography in part because of the demands of Cousteau's scuba expeditions (work that led to basic building blocks, like the Nikonos underwater cameras). Edgerton was of course a brilliantly innovative photographer, and a master storyteller. Despite these precedents, however, learning to produce an effective visual record of fieldwork (or even labwork) is not typically part of a scientist's training. Few scientists are master story tellers. Few learn to illustrate powerfully. And expeditionary learning opportunities are rare.

When language is a barrier, photos and videos take on an even greater importance. This is especially true in a networked world, because the net brings together people without regard to literacy rates or the local vernacular. Yet despite terrific advances in both computer network tools and digital image technologies (see, for example, the digital photography review), it is still a very awkward and painful process to capture a good photographic record of an expedition and share it online. That is true whether you're taking a geology team to Greenland, or your family to Disneyland. Better imaging tools are needed.


2. Plan: An Expedition to Bhutan

— general workplan;
— why Bhutan;
— discussion of work plan.

Our plan is to focus an expedition on improving picture sharing technologies. On the face of it, the Bhutan trip in the first two weeks of March will yield a corpus of 20,000 images and 30 hours of video that document the lives of children who grow up in remote places. This is a demanding body of material, and intrinsically educational: the photos will be published as an online archive that will be launched in partnership with EPALS.COM (a classroom network of 4m kids and teachers in over 190 countries around the world), and selected images will be used in a book, Growing Up In Bhutan, that will teach children around the world about life in one of the world's extraordinary cultures and raise money to build and assist schools in Bhutan and other developing nations.

Specifically, our plan comprises three tiers of work:

  1. Work on digital photography research problems:
    1. Better capture techniques, including use of metadata like GPS, etc;
    2. Better archiving/retrieval techniques (how can one most effectively access a corpus of tens of thousands of images online?);
    3. Better software for producing presentations from these photobanks (how can one best create a "Ken Burns"-style photo documentary?).

  2. Produce several specific educational results as proofs-of-concept:
    1. A demonstrable web photo archive;
    2. A web-accessible photodocumentary;
    3. A book, Growing Up In Bhutan, part of a new educational series for children.

  3. Use the focused work to further develop two programs:
    1. MIT's Global Opportunities effort on expeditions
    2. Friendly Planet's educational book series

Let's briefly describe why Bhutan is of special interest, then discuss each work area in turn.

Why Bhutan?

Bhutan is a small kingdom nestled in the Himalayas between Nepal, India, and Tibet (China). It is the size of Switzerland (40,000 sqkm), but with an official population of 600,000 it is the only underpopulated country in Asia. Bhutan is famous as the last "shangri la": the ecology is pristine (it's been justifiably called the Galápagos of the Himalaya, due to the large number of endemic species); the rich culture is strikingly intact (Bhutan famously did not bring in television until 1999); few tourists go (perhaps 5,000 each year); and the king and ministry have a stated economic plan built on the concept of ensuring "gross national happiness." The government is simultaneously managing the needs of a developing and largely rural nation, cultivating one of the world's most exquisite surviving cultures, and yet modernizing and interfacing in nimble and forward-looking ways.

The Bhutanese educational system is very much in need of resources (more schools need to be built, better libraries and facilities are required, etc), and there is a stated plan to equip high schools with internetworked computers. Most children in Bhutan speak English: the native language is dzongka, but English is the medium of instruction in schools. The biggest educational dilemma faced in Bhutan is that very bright and very well educated students are produced but the regional economy has little industry or demand for such well-trained people. This is an interesting problem to have, but one that will be increasingly faced by any developing country that invests well in education.

All of these qualities make Bhutan an extraordinary site for our proposed work. However, Bhutan is understandably very selective in dealing with any outside projects. The government requires official approval before any outsiders conduct a project, and exercises special scrutiny in the case of any films, television programs, or books. We are fortunate in that we have top-level government contacts (the Minister of Education, the Minister of Communication, and the heads of several other branches are already assisting us), and members of the royal family, including Ashi (princess) Khendum Dorji, have helped us manage securing approval and travel. There are some very practical consequences to this. For one thing, we have been given carte blanche access to the school system (and last fall, met with a number of principals, headmasters and students across the country). For another, the not-so-simple logistics of getting into and out of Bhutan are manageable. (One can only enter the country by working with an internal government-approved tour agency, and normally one pays a surcharge of $200/person per day to visit; we already have a working relationship with the royal travel agency, Chhundu, and because this is a government approved effort, the normal taxes and charges will be waived).

2A. Research Problems in Field Photography

An example of the current state of the art in expedition photography might be seen in a slideshow documenting a joint project of the University of Hawaii and MIT. The work here involved a very rare species of plant, Silene Hawaiiensis (just a few hundred individuals exist, living solely in the desert zone around the Kilauea craters), and an innovative sensing system to monitor the ecology around them.

You'll notice a simple browser window (the digital version of a contact sheet) that displays thumbnail images above, and a large, captioned photo below. The browser is a minimalist design, and allows one to zoom, start a simple slideshow, etc. Detailed shot data (which can include GPS information, HTML directives embedded in the captions, etc) can be toggled on or off (press i). Images can be zoomed (press + or ), and the highest resolution image can be opened in a separate window (double click on a thumbnail image).

It should be possible to capture, caption, and narrate these images in the field, while thoughts are fresh. But at the moment, even to do such basic things as inserting a caption or narration or GPS stamp into the header of an image, and making the stuff browseable on the web, is surprisingly awkward. There are few good tools for the job. (The author wrote the software to produce the Hawaii/MIT slideshow, primarily so the work could be shared with colleagues at the two universities). For instance, notice that some of the image captions contain hot links (to the photographer, or to academic references related to the image). This is simply the result of putting html in the image headers, and rendering it appropriately in a web view -- a capability that should be utterly commonplace. Absent a fulltime publishing and design partner, this sort of modest photojournal considerably exceeds what one typically sees from a field scientist.

Moreover, there are particularly interesting things that one might like to do given that considerable camera data is present in the image header. For instance, since the state of the lens is known (often including the calculated subject distance), it should be possible to compute the size of an object in focus. This means you could overlay a computed ruler to measure how big things are in a scene, something many scientists would like to be able to do. Filtering a photo archive to find only the close-up shots or the panoramas would be interesting. These fall in the category of tempting "what if..." questions.

GPS information needs special mention. Knowing precisely when and where every image was taken is a small step for computer science, but a giant leap for field photography. For example, architects and planners photographing buildings or settlements ought to be able to precisely date and locate photographs taken over the years. GPS tagging is the solution. Yet although the JPG/EXIF header actually includes fields for GPS data, there are almost no cameras that use it (and no camera that we are aware of has an embedded GPS sensor). A few experimental cameras have been built to use an external GPS receiver to tag images, and tie the online imagery to browseable maps. A handful of video+GPS tools exist for production cinematographers. This is clearly a capability that every field photographer wants, a vital way to anchor images to time and place, and a great hook for further educational development. But it is essentially undeveloped territory. We are beginning an assault on the problem, and have recently shot 15,000 images, all of them GPS coded, across the kingdom of Cambodia. We will do the same in Bhutan. By synchronizing camera clocks with GPS atomic time, it is possible for a field photographer to shoot with many cameras, log the GPS information, and stamp the GPS data into the imagery as a post-processing step.

We do not yet have a fully adequate means of searching large image archives using the metadata or image contents. Many possible techniques for doing this have been researched, including using vision and scene analysis. Few have been put into practice in a commonly available form. As a result, when a field team comes back with a large corpus of images, they are archived on a shelf, or in a digital shoebox of some haphazard sort. Moreover, there is at present no way to "deposit" a digital image into the internet for permanent archiving and retrieval. (This might be seen as akin to the situation with the Library of Congress before legislation was passed encouraging the deposit of all books into the library). While internet search services like Google do make it possible to look up images, the lack of use of metadata or other coherent indexing and tagging techniques fundamentally hamstrings these systems. Finally, note that current prices of a terabyte of storage and server are well under $10,000 and dropping rapidly. The missing link is not the capture devices, nor the storage, nor the network: It is being able to store one's photo archives in a framework that affords this kind of browseability.

The problem of producing a compelling presentation has long been a bugbear for just about everyone. When we alluded to Ken Burns, we were referring to educational films like The Civil War, in which simple but powerful photodocumentary techniques were used. The "Burns" visual language includes slowly panning or zooming across a photograph, crossfades, and the combination with a very effective voice-over narration and soundtrack. All of these things can in principle be done now on any current internet-connected computer. What's missing is an authoring system and browser to make this possible. The tools to do this well are not currently in place. We plan to begin this work and will demonstrate one solution.

2B. Educational Results from the Bhutan Expedition

The Bhutan expedition needs to produce focused results in six to nine months, including a web archive, a web-accessible photodocumentary, and a book. These results will be de facto first solutions to the problems outlined above.

Here are the steps we'll take:

Raw Material: Photographic material will include a mix of sources: a large number (> 10k) of digital photos, taken with professional digital cameras (Canon D30, and Canon 1-D); an equal or greater number of analog film photos (Fuji Velvia 35mm slide film, Canon EOS 1-v, scanned at 4000dpi with Nikon LS-4000ed scanners), and a quantity of mini-DV video (about 20-30 hours). This raw material will be archived on a new multiterabyte store that has been donated to MIT by Brewster Kahle and the Internet Archive (www.archive.org). (It will further be "cloned" on the internet archive).

Student Photographers: Many of the images will be taken by children and students in Bhutan. We will send field kits (laptop, digital camera, shooting instructions) to selected students to do this, and have already identified two students in Bhutan (one a photography student, the other the top high school student in the best high school there) to do this. There are two immediate, urgent reasons to do this. One is to get a real, first-person perspective by engaging students from the region in the documentary work. Two is to begin to define the field kits and processes for dealing with field photography teams, collecting their results, etc. And the longer term reason is to develop steps to equip students in other regions with the means to document and share their culture with other students around the world.

Web Archive: The web archive will include a new browser interface for all the photographs. In addition to the minimal "contact sheet" browsers, the image archive needs to permit searching by subject, time, and place. It will be launched initially via EPALS.COM, which is the world's largest classroom exchange for kids and teachers. EPALS engages an audience of 4m kids and teachers in over 190 countries, and their business includes providing safe email, multilingual translation, and curricular development. For example, EPALS was recently chosen (over Microsoft and AOL) by the State of Arizona to provide school-safe email for the 700k students there. This initial teething exercise with the members of EPALS will provide an exceptional and highly visible testbed. The archive will of course be available to the greater internet audience, but our initial interest lies in applying it for student education. The main challenge here lies in building a new software platform for maintaining and browsing these archives.

Photodocumentary: What we envision here is a voice and music-narrated photo documentary, viewable on the internet. This will likely require developing a new client browser, probably openGL-based, that can access a script that taps URL resources (the narration, the soundtrack, the imagery, and the descriptions and timings of transitions (wipes, dissolves, etc). Our ideal design would function like a screensaver, grabbing the whole display, fetching the contents, and rendering the sequence. Tools like Powerpoint and languages like Flash are both overkill and underkill for this job. The photodocumentary viewer will similarly be available via the web archive. This work is likely to be of immediate interest to individuals at Microsoft, and we will work with Salesin to coordinate that.

Book: The book Growing Up In Bhutan will be produced as a direct byproduct of this work. It will be the second in a series of photobooks for kids that document what life is like in some of the world's most remarkable places. The first, Growing Up In Cambodia is in preparation now, and there is some background imagery describing a newly donated school system in rural Cambodia here; and an article on the effort here. Gui/Bhutan will contain 200 of the most stunning images drawn from the archive that show some of the facets of life, culture and ecology there. It will raise charitable funds for the Bhutanese school system, along the lines discussed in 2C below.

2C. MIT's GO, and Friendly Planet's Growing Up

Friendly Planet is a new, non-profit company whose mission is to stimulate charities by and for kids, and it is the organization producing the book series. A PDF describing the series in production may be found here. Briefly, the concept is to show what it's like to grow up in interesting regions, using a format redolent of the smash-hit Day in the Life series, but this is a view by and for kids. Each book will contain 200 stunning photographs (many shot by kids in those places) that show a vivid, balanced view of the unique facets of life, culture, and ecology. The books are being planned to spur charity funding: 10,000 copies of Growing Up In Cambodia will be pre-sold to patrons for $100/ea; at least half of the resulting contribution will be used to build new schools in rural Cambodian villages. Friendly Planet is developing partnerships with National Geographic and others to ensure wider charitable reach. The goal is to raise $1m of direct charity input for each country covered; to produce books for three countries per year; and to use the books (and web tie-ins) as instruments to engage kids to learn from each other across great world divides.

MIT GO is a new program launched within MIT to work across departments at MIT specifically to stimulate great expeditionary work. The effort grew from expeditionary efforts funded by DARPA at the Media Lab (directed by Professor Hawley, budgeted at $1m+/year). Media Lab expeditions included the first major scientific expedition to Mount Everest (focusing on the GPS summit survey; remote weather monitoring; climber physiology monitoring; and telemedicine); work on the geology of the eastern shield of Greenland; nordic skiing in Iceland and Norway (involving a new microsensor array to monitor ski kinematics); and others. The idea behind GO is to work across university boundaries on more ambitious scientific field efforts. Universities tend to be compartmentalized. MIT is no exception. But expeditions demand a synthetic array of skills and tools, so by definition they ought to engage a variety of sectors in a university.

The GO program of expeditions will work in affiliation with partners including the Boston Museum of Science and the National Geographic Society to ensure that not only results, but the passion of these scientific enterprises is conveyed to a broader public.

For the proposed Bhutan expedition, all photographs will be donated by MIT/GO to Friendly Planet for its nonprofit educational use. In this way, MIT concentrates on the technological foundations to empower these sorts of field expeditions.


3. Timetable, Budget and Delivery

2/25-3/10: fieldwork in Bhutan (4 people, 1-2 students)

4/1: preliminary image archive on line

5/1: full raw digital image archive on line; book design underway.

6/1: full raw digital and on Internet Archive

8/1: first version of photodocumentary

8/20: possible followup trip to Bhutan (2 people)

9/15: launch image archive and photodocumentary with EPALS

11/1: Growing Up In Bhutan published

Budget

The budget summary ($175k, 1 year project) is:

Travel, Equipment, Communications:
  $25k  travel     (5 ppl @ $3k airfare, $10k ground expenses)
  $20k  photo     (still/video cameras, film/processing/printing)
  $20k  digital     (scanners/pc, gps, software)
  $ 5k  com & is     (telephone, MIT IS, print/publishing service)
Personnel:
  $60k  Newell    (50%, 12 months)
  $15k  Choi    (50%, 12 months)
  $30k  2 students    (50%, 12 months)

Notes:

Travel assumes 5 person team (Hawley, Choi, Newell, Salesin, Student) economy from Boston via Kathmandu or Bangkok to Bhutan, plus ground expenses for two weeks. (A second possible trip to Bhutan is not budgeted here).
Photo equipment includes new (up to date) video gear, two high-end Canon 1-D digital cameras, lenses, supplies (film, processing, printing, high density 1Gb flash, portable 30Gb disks).
Digital equipment includes film scanners (Nikon LS-4000ed with dedicated scanning stations), printers (two archival Epson printers and media, one large format), gps receivers, software.
Com & IS includes telephone, MIT information services, external print/publishing.

Personnel roles: Newell will produce the expedition and administer the program; Choi will manage the photography and post-production work; students will assist with fieldwork, programming. Hawley's salary is covered by MIT. Salesin's salary is covered by Microsoft.

Delivery

The key delivery steps, roughly monthly, involve: assembling the raw image archive (both locally and in the Internet Archive); producing the refined (browseable) image archive and photodocumentary; producing the book; launching the educational materials with EPALS.

We have planned personnel salary coverage for 12 months, although the current deadlines only extend through 9 months. Deadlines in this sort of plan have a natural elasticity.


4. Summary and Impact

The proposed plan, Learning from Pictures, uses a photoexpedition to Bhutan as the driver to explore research problems in digital photography; and to build specific artifacts (web site, photodocumentary, book Growing Up In Bhutan) as proof of the concepts. Proceeds from sales of books and other artifacts will directly benefit rural schools in Bhutan and help to create the foundation of an educational book series for kids. (We expect the book to generate $500k-$1m of direct input to the Bhutanese schools). The technological work will improve the foundation for taking and sharing great expeditionary photography, and will be used by MIT's new GO program to empower expeditionary work across other institute departments. Special partnerships (with the Internet Archive, EPALS, Friendly Planet, and the Royal Government of Bhutan), as well as MIT GO's affiliation with the Boston Museum of Science, will ensure that the work done here reaches many educational beneficiaries.