Chapter Three: Achieving the Computational Capacity
of the Human Brain
1. Gordon E. Moore, “Cramming More Components onto Integrated Circuits,” Electronics 38.8 (April 19, 1965): 114–17, ftp://download.intel.com/research/silicon/moorespaper.pdf.
2. Moore’s initial projection in this 1965 paper was that the number of components would double every year. In 1975 this was revised to every two years. However, this more than doubles price-performance every two years because smaller components run faster (because the electronics have less distance to travel). So overall price-performance (for the cost of each transistor cycle) has been coming down by half about every thirteen months.
3. Paolo Gargini quoted in Ann Steffora Mutschler, “Moore’s Law Here to Stay,” ElectronicsWeekly.com, July 14, 2004, http://www.electronicsweekly.co.uk/articles/article.asp?liArticleID=36829. See also Tom Krazit, “Intel Prepares for Next 20 Years of Chip Making,” Computerworld, October 25, 2004, http://www.computer world.com/hardwaretopics/hardware/story/0,10801,96917,00.html.
4. Michael Kanellos, “ ‘High-rise’ Chips Sneak on Market,” CNET News.com, July 13, 2004, http://zdnet.com.com/2100-1103-5267738.html.
5. Benjamin Fulford, “Chipmakers Are Running Out of Room: The Answer Might Lie in 3-D,” Forbes.com, July 22, 2002, http://www.forbes.com/forbes/2002/0722/173_print.html.
6. NTT news release, “Three-Dimensional Nanofabrication Using Electron Beam Lithography,” February 2, 2004, http://www.ntt.co.jp/news/news04e/0402/040202.html.
7. László Forró and Christian Schönenberger, “Carbon Nanotubes, Materials for the Future,” Europhysics News 32.3 (2001), http://www.europhysicsnews.com/full/09/article3/article3.html. Also see http://www.research.ibm.com/nanoscience/nanotubes.html for an overview of nanotubes.
8. Michael Bernstein, American Chemical Society news release, “High-Speed Nano-tube Transistors Could Lead to Better Cell Phones, Faster Computers,” April 27, 2004, http://www.eurekalert.org/pub_releases/2004-04/acs-nt042704.php.
9. I estimate a nanotube-based transistor and supporting circuitry and connections require approximately a ten-nanometer cube (the transistor itself will be a fraction of this), or 103 cubic nanometers. This is conservative, since single-walled nanotubes are only one nanometer in diameter. One inch = 2.54 centimeters = 2.54 × 107 nanometers. Thus, a 1-inch cube = 2.543 × 1021 = 1.6 × 1022 cubic nanometers. So a one-inch cube could provide 1.6 × 1019 transistors. With each computer requiring approximately 107 transistors (which is a much more complex apparatus than that comprising the calculations in a human interneuronal connection), we can support about 1012 (one trillion) parallel computers.
A nanotube transistor–based computer at 1012 calculations per second (based on Burke’s estimate) gives us a speed estimate of 1024 cps for the one-inch cube of nanotube circuitry. Also see Bernstein, “High-Speed Nanotube Transistors.”
With an estimate of 1016 cps for functional emulation of the human brain (see discussion later in this chapter), this gives us about 100 million (108) human-brain equivalents. If we use the more conservative 1019 cps estimate needed for neuromorphic simulation (simulating every nonlinearity in every neural component; see subsequent discussion in this chapter), a one-inch cube of nanotube circuitry would provide only one hundred thousand human-brain equivalents.
10. “Only four years ago did we measure for the first time any electronic transport through a nanotube. Now, we are exploring what can be done and what cannot in terms of single-molecule devices. The next step will be to think about how to combine these elements into complex circuits,” says one of the authors, Cees Dekker, of Henk W. Ch. Postma et al., “Carbon Nanotube Single-Electron Transistors at Room Temperature,” Science 293.5527 (July 6, 2001): 76–129, described in the American Association for the Advancement of Science news release,“Nano-transistor Switches with Just One Electron May Be Ideal for Molecular Computers, Science Study Shows,” http://www.eurekalert.org/pub_releases/2001-07/aaft-nsw062901.php.
11. The IBM researchers solved a problem in nanotube fabrication. When carbon soot is heated to create the tubes, a large number of unusable metallic tubes are created along with the semiconductor tubes suitable for transistors. The team included both types of nanotubes in a circuit and then used electrical pulses to shatter the undesirable ones—a far more efficient approach than cherry-picking the desirable tubes with an atomic-force microscope. Mark K. Anderson, “Mega Steps Toward the Nanochip,” Wired News, April 27, 2001, at http://www.wired.com/news/technology/0,1282,43324,00.html, referring to Philip G. Collins, Michael S. Arnold, and Phaedon Avouris, “Engineering Carbon Nanotubes and Nanotube Circuits Using Electrical Breakdown,” Science 292.5517 (April 27, 2001): 706–9.
12. “A carbon nanotube, which looks like rolled chicken wire when examined at the atomic level, is tens of thousands of times thinner than a human hair, yet remarkably strong.” University of California at Berkeley press release,“Researchers Create First Ever Integrated Silicon Circuit with Nanotube Transistors,” January 5, 2004, http://www.berkeley.edu/news/media/releases/2004/01/05_nano.shtml, referring to Yu-Chih Tseng et al., “Monolithic Integration of Carbon Nanotube Devices with Silicon MOS Technology,” Nano Letters 4.1 (2004): 123–27, http://pubs.acs.org/cgi-bin/sample.cgi/nalefd/2004/4/i01/pdf/nl0349707.pdf.
13. R. Colin Johnson, “IBM Nanotubes May Enable Molecular-Scale Chips,” EETimes, April 26, 2001, http://eetimes.com/article/showArticle.jhtml?articleId=10807704.
14. Avi Aviram and Mark A. Ratner, “Molecular Rectifiers,” Chemical Physics Letters (November 15, 1974): 277–83, referred to in Charles M. Lieber, “The Incredible Shrinking Circuit,” Scientific American (September 2001), at http://www.sciam.com and http://www-mcg.uni-r.de/downloads/lieber.pdf. The single-molecule rectifier described in Aviram and Ratner could pass current preferentially in either direction.
15. Will Knight, “Single Atom Memory Device Stores Data,” NewScientist.com, September 10, 2002, http://www.newscientist.com/news/news.jsp?id=ns99992775, referring to R. Bennewitz et al., “Atomic Scale Memory at a Silicon Surface,” Nanotechnology 13 (July 4, 2002): 499–502.
16. Their transistor is made from indium phosphide and indium gallium arsenide. University of Illinois at Urbana-Champaign news release, “Illinois Researchers Create World’s Fastest Transistor—Again,” http://www.eurekalert.org/pub_ releases/2003-11/uoia-irc110703.php.
17. Michael R. Diehl et al., “Self-Assembled Deterministic Carbon Nanotube Wiring Networks,” Angewandte Chemie International Edition 41.2 (2002): 353–56; C. P. Collier et al., “Electronically Configurable Molecular-Based Logic Gates,” Science 285.5426 (July 1999): 391–94. See http://www.its.caltech.edu/~heathgrp/papers/Paperfiles/2002/
diehlangchemint.pdf and http://www.cs.duke.edu/~thl/papers/Heath.Switch.pdf.
18. The “rosette nanotubes” designed by the Purdue team contain carbon, nitrogen, hydrogen, and oxygen. The rosettes self-assemble because their interiors are hydrophobic and their exteriors are hydrophilic; therefore, to protect their insides from water, the rosettes stack into nanotubes. “The physical and chemical properties of our rosette nanotubes can now be modified almost at will through a novel dial-in approach,” according to lead researcher Hicham Fenniri. R. Colin Johnson, “Purdue Researchers Build Made-to-Order Nanotubes,” EETimes, October 24, 2002, http://www.eetimes.com/article/showArticle.jhtml?articleId=18307660; H. Fenniri et al., “Entropically Driven Self-Assembly of Multichannel Rosette Nanotubes,” Proceedings of the National Academy of Sciences 99, suppl. 2 (April 30, 2002): 6487–92; Purdue news release, “Adaptable Nanotubes Make Way for Custom-Built Structures, Wires,” http://news.uns.purdue.edu/UNS/html4ever/
020311.Fenniri.scaffold.html.
Similar work has been done by scientists in the Netherlands: Gaia Vince, “Nano-Transistor Self-Assembles Using Biology,” NewScientist.com, November 20, 2003, http://www.newscientist.com/news/news.jsp?id=ns99994406.
19. Liz Kalaugher, “Lithography Makes a Co
nnection for Nanowire Devices,” June 9, 2004, http://www.nanotechweb.org/articles/news/3/6/6/1, referring to Song Jin et al., “Scalable Interconnection and Integration of Nanowire Devices Without Registration,” Nano Letters 4.5 (2004): 915–19.
20. Chao Li et al., “Multilevel Memory Based on Molecular Devices,” Applied Physics Letters 84.11 (March 15, 2004): 1949–51. Also see http://www.technologyreview.com/articles/rnb_051304.asp?p=1. See also http://nanolab.usc.edu/PDF%5CAPL84-1949.pdf.
21. Gary Stix, “Nano Patterning,” Scientific American (February 9, 2004), http://www.sciam.com/print_version.cfm?articleID=000170D6-C99F-101E-861F83414B7F0000; Michael Kanellos, “IBM Gets Chip Circuits to Draw Themselves,” CNET News.com, http://zdnet.com.com/2100-1103-5114066.html. See also http://www.nanopolis.net/news_ind.php?type_id=3.
22. IBM is working on chips that automatically reconfigure as needed, such as by adding memory or accelerators. “In the future, the chip you have may not be the chip you bought,” said Bernard Meyerson, chief technologist, IBM Systems and Technology Group. IBM press release, “IBM Plans Industry’s First Openly Customizable Microprocessor,” http://www.ibm.com/investor/press/mar-2004/31-03-04-1.phtml.
23. BBC News, “ ‘Nanowire’ Breakthrough Hailed,” April 1, 2003, http://news.bbc.co.uk/1/hi/sci/tech/2906621.stm. Published article is Thomas Scheibel et al., “Conducting Nanowires Built by Controlled Self-Assembly of Amyloid Fibers and Selective Metal Deposition,” Proceedings of the National Academy of Sciences 100.8 (April 15, 2003): 4527–32, published online April 2, 2003, http://www.pnas.org/cgi/content/full/100/8/4527.
24. Duke University press release, “Duke Scientists ‘Program’ DNA Molecules to Self Assemble into Patterned Nanostructures,” http://www.eurekalert.org/pub_releases/2003-09/du-ds092403.php, referring to Hao Yan et al., “DNA-Templated Self-Assembly of Protein Arrays and Highly Conductive Nanowires,” Science 301.5641 (September 26, 2003): 1882–84. See also http://www.phy.duke.edu/~gleb/Pdf_FILES/DNA_science.pdf.
25. Ibid.
26. Here is an example of the procedure to solve what’s called the traveling-salesperson problem. We try to find an optimal route for a hypothetical traveler among multiple cities without having to visit a city more than once. Only certain city pairs are connected by routes, so finding the right path is not straightforward.
To solve the traveling-salesperson problem, mathematician Leonard Adleman of the University of Southern California performed the following steps:
1. Generate a small strand of DNA with a unique code for each city.
2. Replicate each such strand (one for each city) trillions of times using PCR.
3. Next, put the pools of DNA (one for each city) together in a test tube. This step uses DNA’s affinity to link strands together. Longer strands will form automatically. Each such strand represents a possible route of multiple cities. The small strands representing each city link up with each other in a random fashion, so there is no mathematical certainty that a linked strand representing the correct answer (sequence of cities) will be formed. However, the number of strands is so vast that it is virtually certain that at least one strand—and probably millions—will be formed that represents the correct answer.
The next steps use specially designed enzymes to eliminate the trillions of strands that represent wrong answers, leaving only the strands representing the correct answer:
4. Use molecules called “primers” to destroy those DNA strands that do not start with the start city, as well as those that do not end with the end city; then replicate the surviving strands, using PCR.
5. Use an enzyme reaction to eliminate those DNA strands that represent a travel path greater than the total number of cities.
6. Use an enzyme reaction to destroy those strands that do not include city 1. Repeat for each of the cities.
7. Now, each of the surviving strands represents the correct answer. Replicate these surviving strands (using PCR) until there are billions of such strands.
8. Using a technique called electrophoresis, read out the DNA sequence of these correct strands (as a group). The readout looks like a set of distinct lines, which specifies the correct sequence of cities.
See L. M. Adleman, “Molecular Computation of Solutions to Combinatorial Problems,” Science 266 (1994): 1021–24.
27. Charles Choi, “DNA Computer Sets Guinness Record,” http://www.upi.com/view.cfm?StoryID=20030224-045551-7398r. See also Y. Benenson et al., “DNA Molecule Provides a Computing Machine with Both Data and Fuel,” Proceedings of the National Academy of Sciences 100.5 (March 4, 2003): 2191–96, available at http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=12601148; Y. Benenson et al., “An Autonomous Molecular Computer for Logical Control of Gene Expression,” Nature 429.6990 (May 27, 2004): 423–29 (published online, April 28, 2004), available at http://www.wisdom.weizmann.ac.il/~udi/ShapiroNature2004.pdf.
28. Stanford University news release, “ ‘Spintronics’ Could Enable a New Generation of Electronic Devices, Physicists Say,” http://www.eurekalert.org/pub_releases/2003-08/su-ce080803.php, referring to Shuichi Murakami, Naoto Nagaosa, and Shou-Cheng Zhang, “Dissipationless Quantum Spin Current at Room Temperature,” Science 301.5638 (September 5, 2003): 1348–51.
29. Celeste Biever, “Silicon-Based Magnets Boost Spintronics,” NewScientist.com, March 22, 2004, http://www.newscientist.com/news/news.jsp?id=ns99994801, referring to Steve Pearton,“Silicon-Based Spintronics,” Nature Materials 3.4 (April 2004): 203–4.
30. Will Knight, “Digital Image Stored in Single Molecule,” NewScientist.com, December 1, 2002, http://www.newscientist.com/news/news.jsp?id=ns99993129, referring to Anatoly K. Khitrin, Vladimir L. Ermakov, and B. M. Fung, “Nuclear Magnetic Resonance Molecular Photography,” Journal of Chemical Physics 117.15 (October 15, 2002): 6903–6.
31. Reuters, “Processing at the Speed of Light,” Wired News, http://www.wired.com/news/technology/0,1282,61009,00.html.
32. To date, the largest number to be factored is one of 512 bits, according to RSA Security.
33. Stephan Gulde et al., “Implementation of the Deutsch-Jozsa Algorithm on an Ion-Trap Quantum Computer,” Nature 421 (January 2, 2003): 48–50. See http://heart-c704.uibk.ac.at/Papers/Nature03–Gulde.pdf.
34. Since we are currently doubling the price-performance of computation each year, a factor of a thousand requires ten doublings, or ten years. But we are also (slowly) decreasing the doubling time itself, so the actual figure is eight years.
35. Each subsequent thousandfold increase is itself occurring at a slightly faster rate. See the previous note.
36. Hans Moravec, “Rise of the Robots,” Scientific American (December 1999): 124–35, http://www.sciam.com and http://www.frc.ri.cmu.edu/~hpm/project. archive/robot.papers/1999/SciAm.scan.html. Moravec is a professor at the Robotics Institute at Carnegie Mellon University. His Mobile Robot Laboratory explores how to use cameras, sonars, and other sensors to give robots 3-D spatial awareness. In the 1990s, he described a succession of robot generations that would “essentially [be] our off-spring, by unconventional means. Ultimately, I think they’re on their own and they’ll do things that we can’t imagine or understand — you know, just the way children do” (Nova Online interview with Hans Moravec, October 1997, http://www.pbs.org/wgbh/nova/robots/moravec.html). His books Mind Children: The Future of Robot and Human Intelligence and Robot: Mere Machine to Transcendent Mind explore the capabilities of the current and future robot generations.
Disclosure: The author is an investor in and on the board of directors of Moravec’s robotics company, Seegrid.
37. Although instructions per second as used by Moravec and calculations per second are slightly different concepts, these are close enough for the purposes of these order-of-magnitude estimates. Moravec developed the mathematical techniques for his robot vision independent of biological models, but similarities (between Moravec’s algorithms and those performed biologically) were noted after the fact. Functionally, Moravec’s comp
utations re-create what is accomplished in these neural regions, so computational estimates based on Moravec’s algorithms are appropriate in determining what is required to achieve functionally equivalent transformations.
38. Lloyd Watts, “Event-Driven Simulation of Networks of Spiking Neurons,” seventh Neural Information Processing Systems Foundation Conference, 1993; Lloyd Watts, “The Mode-Coupling Liouville-Green Approximation for a Two-Dimensional Cochlear Model,” Journal of the Acoustical Society of America 108.5 (November 2000): 2266–71. Watts is the founder of Audience, Inc., which is devoted to applying functional simulation of regions of the human auditory system to applications in sound processing, including creating a way of preprocessing sound for automated speech-recognition systems. For more information, see http://www.lloydwatts.com/neuroscience.shtml.
Disclosure: The author is an adviser to Audience.
39. U.S. Patent Application 20030095667, U.S. Patent and Trademark Office, May 22, 2003.
40. The Medtronic MiniMed closed-loop artificial pancreas currently in human clinical trials is returning encouraging results. The company has announced that the device should be on the market within the next five years. Medtronic news release, “Medtronic Supports Juvenile Diabetes Research Foundation’s Recognition of Artificial Pancreas as a Potential ‘Cure’ for Diabetes,” March 23, 2004, http://www.medtronic.com/newsroom/news_2004323a.html. Such devices require a glucose sensor, an insulin pump, and an automated feedback mechanism to monitor insulin levels (International Hospital Federation, “Progress in Artificial Pancreas Development for Treating Diabetes,” http://www.hospitalmanagement.net/informer/technology/tech10). Roche is also in the race to produce an artificial pancreas by 2007. See http://www.roche.com/pages/downloads/science/pdf/rtdcmannh02-6.pdf.