Title: Class Note for Camera Culture Date: Feb 12 By: Tom Yeh Review ====== Matt's Summary -------------- 1. What will a camera look like in 10 years, 20 years? * Continuous trend - smaller - more pixels - more storage * New modalities - other light spectrum - other sensors * New forms Wand, sheet, - implants - retinal * New collaboration - Environmental distribution of the cameras - Pervasive networks - provides new perspectives in time and space 2. What ill be the dominant platform and why? There is the consensus that cellphones will be the dominant platform at present and in near future. After cellphones, the most promising candidate of the next dominant platform is our human body. Discussion ---------- Ramesh: What is the largest camera company today? Not Canon or Nikkon. The largest camera company is Nokia. The Swiss army knife is a good analogy for the ultimate portable digital device designed to combine the functionalities of a myriad of devices into a single device. However, each functionality is only a trimmed down version of the functionality provided by the original single-purpose device. Just as the scissors in a swiss army knife look like a toy when compared to real scissors, cameras on cellphoness still can not rival real cameras in many aspects. Fredo: Serious photographers who need all the bells and whistles of powerful cameras would never be satisfied with handicapped cameras on cellphones. But most people probably don't care about the difference. Ramesh: Consumer-grade cameras may disappear one day. High-end phones start appearing in newer cellphone models. For example, one of the new Sony Ericson phone sports a 7-meg digital cameras with many wheels and whistles like auto-focus, image stabilizer...etc. Brandon: Media lab's bottle of soap project is a reconfigurable device that changes its function based on how the user holds it. The device can behave like a camera, phone, remote control, pda or a gampad. The basic interface across these devices shares the same amount of complexity. (See http://obm.media.mit.edu) Fredo: Implant-based sensing, if used with the same optic as human eyes, is still limited to what human eyes can see. Doug: A sheet-like camera can be inserted into our eyes like contact lenses. Such camera can capture continuous streams of images and provide augmented VR. Just as people are tired of putting into contact lenses everyday and opt for laser corrective surgery, we may one day choose to surgically implant the camera into our eyes. Sirus: A cameron captures each pixel. Camera is the plural of cameron, capable of capturing multiple pixels. Camerons is a collection of cameron that captures a single pixel. Ramesh: In Japan, there is a research group working on a contraption that can translate lights to tactile signals. The contraption is an array of electros that can be placed on the forehead of a blind person. A camera also mounted on the forehead observes the world in low resolution. The pattern captured by the camera triggers a particular firing pattern of the eletros, creating tactile output sufficient for an illusion of object movement that can be interpreted and understood by a blind person with some training. This week's slides ================== Jack Tumblin's Questions ------------------------ When we look at a water bottle on the table, we can see lots of things, such as the incident lighting, the ambiance, the context...etc. But a photograph of the same water bottle lose a lot of information about the water bottle we can see with our eyes. Humans care about mid-level cues, such as the material and identity of the observed object. In contrast, machines perceive only low-level pixels. The big question is whether we can build cameras that can capture richer low-level data to allow human-like mid-level vision easier to achieve? Some researchers applied image processing on a large collection of photos of museum painting and uncovered hidden features, after illumination was taken away, that lead to the finding of Monet's drawing hidden other paintings. Incredibly rich people with terminal diseases pay to have their bodies frozen and preserved in cryogenic states. They hope one day medicine will become so advanced that they can be revived and cured of their diseases. Similarly, can objects be *frozen* in such a way that sufficient details about the objects are preserved and people in the future can analyze these objects using futuristic analytical tools? Archive technology is a huge field. Museums are looking for effective methods to preserve and archive their collections not only in 2D, but also in 2.5D (oil paintings) and 3D (statues). Near Infrared Imaging --------------------- Near infrared imaging enables cameras to look beyond the visible spectrum. Near infrared images of $5 and $20 bills contain patterns invisible to human eyes. Fredo: Full-spectrum painting can produce currencies with unique patterns under different light. These currencies would be difficult to forge using traditional ink. Quinn: New retinal scanners can choose different wavelengths depending on the target. For example, some wavelengths are absorbed more by blood but not by tissues, making them good candidates for detecting blood. Near IR is good for eye examination because our eyes do not respond to it (we can't see) and because near IR can reach deeper. Ramesh: How can we beat traffic cameras? Traffic cameras installed at automatic tollbooth use high speed IR flash to capture images of the license plate of violators. There are three ways to beat them: 1. Overlay the license plate with a film that blocks IR but let visible light pass. 2. Install a flash-triggered auxiliary flash next to the license plate. This auxiliary flash can detect the flash of the IR camera and can fire within milliseconds. This reaction can create glare on the plate, rendering the numbers virtually invisible. Such device can be installed on celebrities such as Britney Spears to blind the cameras of paparazzi. Flash-triggered auxiliary flashes have been already used in photography, and are popular in Japan. Light sensors built into an auxiliary flash detects the firing of the main flash and triggers the flash immediately to provide synchronous lighting. Fredo and Quin: Theaters have developed technologies to detect cameras that may be filming the performance and shoot rays directly at these cameras to blind them. 3. Spray retro-reflective paint on the license plate. Retro-reflectiveness is property of a reflective surface that allows an incoming light to be bounced right back at the same direction it comes in. Retro-reflectiveness ------------------- A surface is retro-reflective if an incoming ray bounces back in the same direction it came in. Retro-reflective paints are widely applied in road signs and road markers so that lights from the headlight of our cars can be bounced right back to us to give us a good visual of these signs and markers. Moon is retro-reflective. During a full moon day, the earth is in-between the moon and the sun. If the sphere of the moon is made of regular reflective material, the light from the sun should fall out of the edge of the moon. However, the moon looks like a shining disk, because the moon behaves retro-reflectively. In restaurant or clothing retailers we often find installation of mirrors on the wall and corner to create an illusion of spaciousness. Standing in a corner where two mirrors are joined at a 90 degree angle, we will see three images of us---one in each of the mirror and the third at the corner. The image at the corner does not have the left-and-right flip, and it stays there even if we move around. Photography: Full of trade-offs ------------------------------ Goal: exhibit sufficient amount of light Why do we need more light? Because more light gives us a better estimate of how bright a particular point in the scene is. On a CCD, each light sensor acts like a bin collecting the number of photons falling into the bin during the exposure. The longer the exposure, the more the photons the bin can receive and more accurate the estimate of the amount of light. If the exposure time is short, a bin may only get a small number of photons, but its neighboring bin may get only a handful more, not enough to distinguish them apart if the amount of noise is high. Available light vs parameter box -------------------------------- The goal is to expand the size of the parameter box as much as possible. A parameter box has four axis: 1. exposure 2. aperture 3. focus distance 4. focal length Let's consider only the first two dimensions for now: exposure and aperture. Sports photos are taken with high aperture and low exposure time. The disadvantage is that the player in a photo is in focus, but everything else is out of focus. It has a very narrow depth of field. This is the effect of *defocus blur*. In the other extreme are night city photos taken with low aperture and high exposure time. It has a wider depth of field. The total amount of light remains the same. The disadvantage is that moving objects would appear blurred. This is the effect of *motion blur*. How do we break this trade-off as much as possible? Dynamic Range ------------- When a scene contains regions under widely disparate lighting conditions, choosing a set of parameters for the entire image is difficult. Some regions may be too bright, while some regions may be too dark. If we fix the parameters ideal for a pixel in a bright region, these parameters are likely to be incorrect for other pixels in a dark region. Dynamic range is a technique designed to alleviate this problem. A popular approach to dynamic range is based on composite of multiple images taken under different parameter setting. Very long exposure ------------------ How did they capture an image under exposure as long as 18 months? Some ideas given by the class are: bad quality film, opaque lenses, very small apertures, but the artist used something different. Matt: The sun tracks actually left traces of weather patterns. Brighter segments may correspond to sunny days whereas darker segments correspond to cloudy days. Fredo: A photographer named Atta Kim took photos of New York streets with 8 hour exposure. The resulting photos appear ghost-town like because all moving elements such as pedestrians and cars are blurred, leaving only the empty and streets. Long-range synthetic aperture photography ----------------------------------------- Synthetic aperture is a technique to create the effect of a magic lens that can accomplish two mutually incompatible goals of large aperture and high depth of field. The idea behind the magic is to use a grid of regular cameras. Images captured by these cameras can be synthesized into a single scene where each part of the scene is in focus. This is achieved by summing up rays seen by individual cameras differently for different points in the scene. Focal length vs viewpoint vs focus ---------------------------------- We looked at examples of portraits of a women taken by a wide angle camera, a standard camera, and a telephoto camera. The portrait taken by the telephoto camera looks prettier and more flattering, but seem cold and unapproachable. The portrait taken by the wide angle camera looks round and less beautiful, but seem warm and friend. Ramesh: We can take a photo a person and apply a learned beauty metric to transform the face in the photo to look more beautiful. Ramesh: Why human brain perceives something beautiful? The dominant theories are the law of average and the law of the golden ratio. Also, new studies suggest that beauty lies in the eyes of an indolent; our brains seem associate beauty with images that requires the least amount of processing. Researchers have shown that a synthetic face image obtained by averaging over a large number of face images does somehow look visually appealing. However, the beautifying effect of averaging may simply be a result of the smoothing of the skin when the images are averaged. Smoothing works like botox, effectively removing imperfections such as wrinkles, freckles, acnes and moles. It may very well be the absence of these imperfect features that explain the attractiveness. What is photography? -------------------- Fredo: Photography should be more than to reproduce experiences of human vision. Human vision have constraints. There are something human vision can't do but photography can. For example, artistic photography has the power of transforming something dull into something visually appealing through clever composition or illumination.