Modern technology owes itself to traditional rules-based programming.¹  Everything from Microsoft Word, Call of Duty, Minecraft, Instagram, smartphone apps, and websites use hardcoded step-by-step instructions to work.² Coders write a list of commands in a computer language, line after line, telling the program precisely what to do. Each one has thousands of lines of code that rigidly script out its behavior.³ There are no surprises with this type of programming; the coder knows what the outcome will be every time  

Behind AI is a flexible new approach to coding where the program writes itself. The programmer writes a code establishing a network of artificial nodes with an input, many layers, and an output. They give it a goal, a way to measure success, and feed it with data; from there, it charts its own path.⁴   

In 2015, Google-backed startup DeepMind Technologies had a breakthrough in training a network to play the game Breakout for Atari. It is like Brickbreaker, where a paddle hits a ball into bricks, with the game ending when it misses the ball. The plan was to see if a network with a goal could learn by playing independently rather than providing it with standard rules-based instructions telling it precisely what to do.

The DeepMind team took a network and gave it the goal of maximizing its score. With that single instruction, it learns how to play by keeping moves that lead to a high score. From there, they made it aware of the game screen and the current score so it could play the way we do. Finally, it plays, and information from the game flows through the network, continually modifying its connections to reflect its learning.

When the DeepMind team turned the breakout playing network on for the first time, it did not know what it was doing. All of its moves were random, and it barely got a score.⁵  The paddle eventually hit the ball into the bricks as it continued to play, and the score went up. Every move leading to an increased score is remembered and stored for future use. After playing hundreds of games, it learns through trial and error, creating a storehouse of moves that allow it to play the game like an expert.

Before it played, it was a fresh network with many nodes organized in layers. Every node from one layer has possible connections with those in the next. Data passes through the layers as it plays, making specific connections and forming unique patterns. For every move it makes, there is a distinctive pattern representing it. 

The connections between every node in the network continually adjust and strengthen toward the pattern that achieves the goal. By exposing a network to vast amounts of data, it fails exhaustively, learning through trial and error. ⁶ It makes mistakes and keeps the patterns that lead to a successful outcome. When it receives that input in the future, it uses the stored pattern that worked in the past. It sounds simple, but after hundreds of games, the Breakout network could play at the same level as an expert human player.  

Networks are different than traditional rules-based programs. The programmer uses code to establish the network, then information passes through and charts its own path. All changes to the connections and patterns in the network coincide with changes to its code. Every adjustment to the network modifies the code, which is why they say it writes itself. The programmer establishes the network but does not know what patterns and subsequent code will emerge to accomplish its goal.

Neural networks are more like children learning, and the programmer is the parent who trains them with data.⁷ The networks behind self-driving vehicles have been trained by physically driving millions of miles and simulating billions more. It is a proven brute force approach whereby continually feeding a network with data will eventually shape it into something functional.

AI and humans use a similar approach to learning, as they both have networks that mold through experience based on underlying goals and drives. Some circuits in the brain are hardwired through evolution, while others are more flexible and take shape through experience. Our circuits that form through experience learn like the breakout playing network discussed above, although they are far more complex. Neural networks must learn the hard way by bumping into things and making mistakes. The nature of a network, whether artificial or biological, is to adjust through experience, molding the network into a framework of understanding around its inborn goal. The human network is more sophisticated in design, but the takeaway is that a network with an objective can learn. Future blogs will take networks to the next level as we venture into how the one between our ears works.