Notes on Fei-Fei Li's 📕The Worlds I See: Curiosity, Exploration & Discovery at the Dawn of AI

I read this book because I attended the talk What We See & What We Value: AI with a Human Perspective by Prof. Li, and wanted to learn more about her research

1. Pins & Needles in D.C.

• She opens the book with a visit to the Smithsonian Air & Space Museum. I think a lot of people, including myself, have felt inspired to learn about STEM topics after a visit to this museum. The other location for me was the Franklin Institute in Philadelphia.

Jawed Karim, CC BY-SA 3.0, via Wikimedia Commons

• Her vision of the ideals of America: “ the dignity of the individual, the intrinsic value of representation, and the belief that human endeavors are best when guided by the many, rather than the few”

2. Something to Chase

• Prejudice against her mother: “ no conspiracy or scandal, just a status quo that revealed its presence through the passive-aggressive remarks…politely but firmly discouraging her ”
• As a child, a lover of books on “ topics as wide-ranging as marine life, robotics, and Chinese mythology ”
• Prejudice against herself: “ It was heavy and jagged, and clawed its way up from a part of me I didn’t even know existed. I didn’t feel discouraged, or even offended. I was angry. It was an anger I wasn’t familiar with—a quiet heat, an indignation like I’d seen in my mother, but unmistakably my own. ”
• Her teachers criticized her for not reading “ textbooks, prep manuals, and selections from the school’s approved reading list ” but instead reading books about “ Marine life, fighter jets, something about UFOs. ” She was criticized for not having the “ discipline to put aside one’s personal interests in favor of the studies that will prove most useful in the years ahead. ”
• Everything changed the first time she saw her mother afraid: “ It was an instantly destabilizing sight, and I wished I could shut my eyes and erase it. Tucked beneath her coat, concealed, but imperfectly so, her hands were trembling. ”

3. A Narrowing Gulf

• An interesting observation about the backgrounds of the early pioneers of AI: “ although many of AI’s founding contributors would go on to explore an eclectic range of fields, including psychology and cognitive science, their backgrounds were almost exclusively centered on mathematics, electrical engineering, and physics itself. ”
• Her family moves to Parsippany, NJ. Hooray, NJ!
• Describes Yann LeCun’s early NJ days at the Bell Labs Holmdel Complex. I’ve worked in two of the Bell Labs buildings, and they had architectures that were fun to just walk around in.

derivative work: MBisanz talkBell_Labs_Holmdel,_The_Oval2.png: *derivative work: MBisanz talkBell_Labs_Holmdel,_The_Oval.jpg: Lee Beaumont, CC BY-SA 2.0, via Wikimedia Commons

• Her mentor has “ a sprawling collection of textbooks and reference volumes that created a rainbow of colored spines facing outward from every wall.”
• In high school, she loved to read on a diverse array of topics: “ But none of it was a distraction from my studies. If anything, by helping me to think more holistically, he reminded me that there’s joy to be found in learning.”

4. Discovering the Mind

• She received criticism for studying physics as an undergraduate instead of “ a lucrative field like medicine, finance, or engineering, and thus an escape from life at society’s margins ”
• As an undergraduate, she found value in the biographies of physicists.
• She didn’t have the computer background that many of her physics classmates had, so she enrolled in a computer science class, which eventually led to a career in computer science.

5. First Light

• NY/NJ firms recruited her during the dot.com boom, but she didn’t want to give up science, and also wanted to leave the bad weather of NJ for the good weather of Caltech in Southern California. This reminded me of Richard Feynman’s “Surely You’re Joking, Mr. Feynman!”: Adventures of a Curious Character where Prof. Feynman tells a story about changing a tire in the cold of Ithaca, NY, and deciding to leave Cornell for Caltech.

6. The North Star

• Researchers were trying to make head-way in artificial intelligence by studying how the human brain works. Her specialty—her “North Star”— was vision: it was crucial to how humans understand the world, and researching human vision would unlock machine vision, and thus machine intelligence.
• Many researchers focused on the algorithms more than the training data, but she believed that the training data was just as important, thus inspiring the ImageNet project.

7. A Hypothesis

• She describes science as “ an incremental pursuit, but its progress is punctuated by sudden moments of seismic inflection—not because of the ambitions of some lone genius, but because of the contributions of many, all brought together by sheer fortune.” The description was similar to the theme of Walter Isaacson’s _The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution _book on the history of computer science.
• A chance conversation between her graduate student and a masters student saved the ImageNet project from insolvency, by introducing her team to crowdsourcing, specifically Amazon Mechanical Turk.
• The decision to leave her professorship at Princeton, where she did her undergraduate degree, for Stanford, is interestingly underdescribed in the book. When she gave the talk What We See & What We Value: AI with a Human Perspective, that decision both filled the room and wasn’t talked about at all.

8. Experimentation

• The ImageNet data didn’t attract enough interest on its own—using the data to evaluate models in competitions, and examine progress over time, made it valuable. You see the same in benchmarks today.
• GPUs, used to accelerate computer graphics in gaming, can be repurposed to process the mathematics used in neural networks. The rise of gaming enabled the rise of deep learning.
• Neural networks were not used much. But the rise of large amounts of traning data like ImageNet, and the rise of large amounts of available compute through GPUs, made deep learning much more useful than in the past, and neural networks replaced many existing machine learning methods.

9. What Lies Beyond Everything?

• Suddenly everything became neural networks, replacing the machine learning algorithms
• Computers can become better than humans at image classification, because they can have much more training data on the types of objects to be classified.

10. Deceptively Simple

• In the medical field, it’s difficult to get training data due to liability and privacy issues, and medical procedures are difficult to quantify or classify. This make machine learning progress in medicine slower than in other fields.
• She describes getting medical studies approved as requiring “ finesse and a kind of diplomatic savvy ” which seems a polite way of saying that it’s somewhat too restrictive.

11. No One’s to Control

• After big corporate research labs declined in the ’90s and ’00s, university labs became be the loci of research, but in modern machine learning, that is no longer true.
• She writes about AGI: “ I couldn’t pinpoint when the term entered the lexicon, but I’d certainly never heard it used in a computer science department. After all, such ‘general’ intelligence had been the whole point of AI since its inception; the fact that we still had such a long way to go didn’t mean we’d set our sights any lower. To the ears of researchers like us, the new term sounded a bit superfluous. But it was catchy, and it made the ultimate ambitions of our field clear to outsiders. ”
• She muses that, for all the talk about AI, she still needed to manually prepare bottles of baby formula.
• She is concerned that her students just want to blitz the latest papers, and urges them to sit down and read the fundamental books. She specifies Russell, Norvig, Minsky, McCarthy, Winograd, Hartley, Zisserman, and Palmer as particularly worthy of study.
• One reason larger companies can do particularly well in AI research is that they have a lot of training data.
• She speaks at a congressional hearing, and her mother is very proud: “ Think about how desperately people around the world long for this kind of thing. An open hearing. Public dialogue between leaders and citizens.”
• She compares the current state of AI to pre-Newtonian physics. Researchers are measuring new phenomena, but there is no formal unified model yet.