A History of Artificial Intelligence — Part 2: The Machine Learning Revolution and the Road to OpenAI
This is Part 2 of a three-part series tracing the history of artificial intelligence. Part 1 covered AI’s philosophical origins, the Dartmouth Conference, and the AI winters. Part 2 picks up with the shift toward machine learning and traces the path to the founding of OpenAI.
A New Approach: Learning From Data Instead of Rules
By the end of Part 1, AI research had hit a wall. The symbolic, rule-based approaches of expert systems, sometimes retroactively called “Good Old Fashioned AI”, involved pre-programming knowledge and rules directly into a system. These approaches were brittle and difficult to scale. A quieter alternative had existed since the field’s earliest days but had never become mainstream: machine learning.
In machine learning, instead of a human programmer writing the rules, the program learns the rules itself from data. Arthur Samuel coined the term “machine learning” itself and was the first to apply the technique, building a self-learning checkers program in 1959 that ran on IBM mainframes. The idea was decades ahead of the computing power needed to make it practical.
The foundational building block for this approach was the artificial neural network, loosely inspired by the structure of the brain. An early version, the Perceptron, was a simple neuron-like model that learned to classify inputs. A 1969 analysis by Minsky and Papert highlighted the perceptron’s limitations, triggering a temporary retreat from neural network research — one of the contributing causes of the AI winters discussed in Part 1.
Backpropagation: The Algorithm That Made Deep Learning Possible
The technique that would eventually revive neural networks was backpropagation. It was a method for training multi-layered networks by calculating how much each connection contributed to an error, then adjusting it accordingly. Backpropagation allowed multi-layer neural networks to adjust their weights to minimize errors, enabling these networks to automatically learn useful internal representations of data.
In the above diagram, the blue forward pass carries activations left to right through the input layer, two hidden layers, and the output layer, with all the inter-neuron weight connections visible. The amber loss box on the right computes the difference between the predicted output and the true label. The coral dashed backward pass then carries the error gradient right to left, and the ∂L/∂w pills mark the points at each layer boundary where weights get adjusted using the chain rule.
In 1986, Geoffrey Hinton, David Rumelhart, and Ronald Williams developed and popularized backpropagation, making the training of multi-layered neural networks genuinely feasible for the first time. In 1989, Yann LeCun provided one of the first practical demonstrations of the technique at Bell Labs, combining convolutional neural networks with backpropagation to read handwritten digits, a system that would later be used to process handwritten numbers on bank checks.
The timing was bittersweet. This breakthrough arrived just as the second AI winter (roughly 1985 to the early 1990s) was setting in, as overly optimistic claims about AI’s “immediate” potential had broken expectations and angered investors, pushing the phrase “artificial intelligence” toward something close to pseudoscience status in some circles. Backpropagation’s true potential would have to wait.
The 1990s: Quiet Progress Beneath the Surface
Even during the chill of the AI winter, important work continued. Throughout the 1990s, Yann LeCun pioneered convolutional neural networks (CNNs), laying the technical foundation for modern computer vision. This is the technology that today powers everything from photo tagging to facial recognition. Researchers also developed the support vector machine, an effective system for mapping and classifying similar data, broadening the machine learning toolkit beyond neural networks alone.
The decade also delivered AI’s first major public spectacle. In 1997, Deep Blue, a chess program built at IBM, defeated reigning world chess champion Garry Kasparov, relying not on deep learning but on brute-force search combined with expert-crafted evaluation rules. It captured global headlines and reignited public fascination with AI, even though the underlying technology was closer to the symbolic AI of earlier decades than to the neural approaches that would soon take over.
The Data Explosion and the ImageNet Moment
Two ingredients were still missing for deep learning to fulfill its promise: enough data to learn from, and enough computing power to process it. The internet supplied the first. The internet boom of the 1990s and 2000s provided machine learning systems with vast quantities of real-world data to learn from for the first time.
Recognizing this, Stanford professor Fei-Fei Li began building ImageNet in 2006, an enormous database that would eventually contain more than 14 million labeled images, designed specifically to give machine learning systems the structured data they needed to learn effectively. Li summarized the philosophy behind the project simply: data drives learning.
Computing power, the second missing ingredient, arrived through an unlikely source: graphics processing units (GPUs), originally built for video games. By 2011, GPU speeds had increased significantly enough to train convolutional neural networks without the cumbersome layer-by-layer pre-training that had previously been necessary.
These two trends collided in 2012, in what is widely considered the single most important turning point in modern AI history. AlexNet, developed by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, won the ImageNet competition, and its success demonstrated that deep learning had decisive advantages in both efficiency and accuracy over older approaches, sparking the deep learning revolution that defines the current era of AI. The rapid adoption of this feature-learning approach by major technology companies followed almost immediately.
From Vision to Strategy: Deep Learning Spreads
Once deep learning proved itself on image recognition, it rapidly expanded into other domains. In 2014, Ian Goodfellow and colleagues introduced Generative Adversarial Networks (GANs), a technique that enabled AI systems to generate convincingly realistic data, an early ancestor of today’s AI image generators.
Perhaps the most striking demonstration of deep learning’s power came from the world of board games. Unlike older symbolic chess programs that relied on pre-programmed evaluation functions, Google DeepMind’s AlphaGo combined deep learning with Monte Carlo tree search, using a neural network to evaluate which moves were promising rather than relying purely on brute-force calculation. In 2015, AlphaGo defeated the world champion Go player, a feat many AI researchers had assumed was still decades away, since Go’s branching complexity vastly exceeds that of chess. AlphaGo achieved this by playing millions of simulated games against itself, learning optimal strategies through deep reinforcement learning rather than human-programmed rules.
2015: The Founding of OpenAI
It was against this backdrop of accelerating breakthroughs and growing public and academic concern about where increasingly powerful AI systems might lead that a new kind of organization was born. In 2015, Elon Musk, Sam Altman, Greg Brockman, and other co-founders established OpenAI, with an explicit mission to promote safe and open AI development.
The founding represented something distinct from the corporate AI labs that had driven progress up to that point. Rather than developing AI purely as a commercial product, OpenAI was structured around the idea that increasingly powerful AI systems needed an organization explicitly focused on ensuring their benefits were broadly shared, and their risks carefully managed. At the time, few could have predicted just how central this organization would become to the AI story over the following decade.
The stage was now set. Deep learning, massive datasets, GPU computing, reinforcement learning, and several other tools, were all in place. What remained was a breakthrough in how machines processed language itself, one that would lead directly to the creation of the Generative Pre-trained Transformer, and eventually, to ChatGPT.
