If you’ve ever had to do anything creative, you’ve experienced failure. It’s a necessary part of the creative process. But there’s more to it than that.

It turns out that when you look at how some of humanity’s greatest minds think of failure, they don’t think of it as an event to be overcome. They see it as a necessary part of the journey towards mastery.

This post explains why failure isn’t something to fear, but rather something to understand. How can it propel us forwards, to our successes?

Mastery has nothing to do with avoiding failure; rather, it’s about relentlessly striving for more.

When people speak about artists and athletes, they often use black-and-white terms, like “good” or “bad,” “success” or “failure.” Well, that kind of hierarchical language is deeply misguided, because failure can actually help you achieve mastery.

To understand why failure can be such an advantage, first we need to understand mastery. Mastery is about endurance, not perfectionism (which is bound up in how we want others to perceive us) or success (related to particular events). In other words, mastery is the unrelenting pursuit of a goal. Think of it like chasing something that can never be caught; it’s about striving for the impossible.

Digging a little deeper, consider the Archer’s Paradox as a metaphor for the process of mastery. The Archer’s Paradox refers to the idea that the archer will draw her bow and point her arrow in a way that's intended to account for elements that are outside of her control, like weather.

So, in a sense, the archer is constantly striving to control things that cannot be controlled. And like the archer, those of us who seek mastery also try to hit the target despite facing enormous trials over which we have no control

And since mastery is borne of this continual process of unrelenting striving, we shouldn’t even use the word “failure” to describe the difficulties we encounter on the path. Because as long as you keep working past the moment of “failure,” the event becomes a learning experience.

Not to mention, these difficulties (i.e. “failures”) can also be a form of motivation. As playwright Tennessee Williams said, “The apparent failure of a play sends me back to my typewriter that very night, before the reviews are out. I am more compelled to get back to work than if I had a success.”

For most of us, there’s a gap between what we have achieved and what we want to achieve.

Since failure is a misguided concept, how can we talk about unrealized achievement? Well, there’s The Gap, a term which refers to the fissure between what you have achieved and what you can achieve.

For instance, a young Hart Crane encountered The Gap when the esteemed poet Ezra Pound deemed Crane’s work “all egg” and no incubator. He meant that Crane’s poetry didn’t match his high potential. In other words: Crane was in The Gap.

Since anyone can fall into The Gap, what can be done to close it? Well first, start by creating a safe haven, a mental or physical space that protects you from criticism and allows you to take risks.

The playwright August Wilson created his safe haven in a restaurant. One time, while he was scribbling away, a waitress asked whether he was writing on napkins “because it doesn’t count.” In fact, that was precisely the case: Wilson wrote on napkins because it felt safe; it allowed him to keep striving without worrying about criticism.

As you can see, a safe haven can be immensely valuable for unleashing creativity, but this protected space also comes with risks. Because after all, when you shield yourself for too long, you often lose touch with reality.

That’s what happened to Pontormo, a sixteenth-century Florentine fresco painter who spent eleven years working on a portrait in isolation. After the work, neither he nor the painting survived.

The risks of isolation are one reason why, after creating a safe haven, you have to find a way to incorporate criticism and pressure. Composer Leonard Bernstein valued having the pressure of limited time: “To achieve great things, two things are needed: a plan, and not quite enough time.”

For example, the author and neurologist Oliver Sacks was desperate because he hadn’t written anything for months. So, he gave himself an ultimatum: write a book in ten days or commit suicide. Nine days later, guess what happened? Sacks finished his book.

Near wins compel us to confront our limitations and push past them.

Have you experienced that sinking feeling that sets in when you miss the bus by just a few seconds? It can be extremely frustrating, especially if you have somewhere important to be. Although these near wins are excruciating, they’re also necessary, because they push us forward

This has actually been proven by psychologists Daniel Kahneman and Amos Tversky, who found that near wins have a profound effect on our thinking, leading us to obsess over “what might have been.” The psychologists even found that the more frustrating the failure, the more it affects our behavior, motivating us to work harder.

For instance, consider Jackie Joyner-Kersee, who was one-third of a second away from winning gold in the 1984 Olympic heptathlon. Her near win had such a motivating effect that when she returned to the Olympics in 1988, she set a record nobody’s beaten since.

But a near win not only motivates us to achieve great things; it also shifts our perspective, focusing our attention on the process of striving and not the end goal.  

Triathlete Julie Moss may be one of the best examples of this phenomenon. During the 1982 Ironman race, Moss had a six minute lead, but then she collapsed during the last half-mile. Although she could barely control her limbs – much less stand up – she still crawled through the last yards of the race, willing herself to push past the pain.

In other words, Moss’ focus shifted from her original goal (winning the race), to the how of the race – that is, how was she going to push past her pain to cross the finish line?

These kinds of near misses define our lives, because the path to the finish line is rarely a straight one. So we learn to keep moving forward, even when it’s a struggle.

In order to reach your potential, you have to surrender yourself to pain.

Near misses motivate us to push past our frustration and accomplish great things. Pain is another difficulty you may have to overcome on the path to mastery: That is, you have to accept and surrender to pain to reach your potential.

Since everyone experiences tragedies, setbacks and disasters, pain is inevitable. A lady was forced to face this sad fact head-on when her seven dear friends all died within a year.

This was obviously a deeply painful experience for her, but eventually she accepted that death is unavoidable. And once she accepted the fragility of life, she was inspired to find more meaning in each moment she had.

That’s how surrendering works: Once you accept your pain fully, you can finally begin to understand it – and then move forward.

The martial art Aikido is a great metaphor for this process. Aikido is all about using your opponent’s force against them, by absorbing their energy and redirecting it back towards them. Wendy Palmer has mastered this technique, making her one of the most powerful and fearsome Aikido fighters, despite being only 5’5”.

Just as Aikido literally trains participants to absorb hostile energy and transform it, we have to learn how to accept setbacks, surrender to their pain and transform the energy or emotions they provoke.

To that end, it’s remarkable how many great leaders faced painful adversity on the path to mastery. For example, Martin Luther King Jr., now renowned as a great orator, had to overcome a speech impediment. In fact, his verbal tic was so noticeable, he was penalized for it when he studied oratory in Seminary.

How did he overcome his impediment? As MLK put it, “Once I’d made my peace with death, I could make my peace with all else.” In other words, once he accepted the fact that life is fragile and painful, his speech impediment simply disappeared.

To get creative, embody the amateur.

If you want to accomplish great things, you have to allow yourself to experiment and play! This idea gets right at the amateur’s advantage – that is, the benefit of having more experience than a novice, but less than an expert. So, why do amateurs have an advantage over experts? Well, amateurs act for pleasure, not money or career-related reasons – and pleasure goes hand and hand with experimentation, leading to truly original new ideas.

To understand the benefits of experimentation, consider scientists Andre Geim and Konstantin Novoselov, who created the world’s first two-dimensional object (a layer of carbon found in graphite) and consequently won a Nobel Prize. This groundbreaking invention was a product of their “Friday Night Experiments,” which consisted of ludicrous experiments with low probabilities of success.

By stepping into the role of amateurs, the scientists had the freedom to play with ideas – eventually leading to a major breakthrough.

And this playfulness is why amateurs have an advantage over experts. The latter are burdened by the Einstellung effect: once someone has developed rigid routines, they are less likely to rethink what works and come up with new ideas. But this is a faulty mindset, because what worked once might not work again

We’ve seen that experimentation results in innovation, and so can playing in a childlike way. That’s why companies like Mattel encourage employees to play in the office, whether on carpets that look like grass, or in chairs that stimulate space shuttles.

There’s even a scientific connection between creativity and play. One study gave two groups of four-year-olds a toy. The researcher showed the first group how to use the toy, but didn’t show the second group. In the end, the second group spent more time playing with the toy and even discovered hidden features.

Ultimately, the study showed that playing intensified the second group’s curiosity, leading them to find more innovative uses for the toy.

On the path to mastery, cultivate grit.

Grit is the final piece to understanding the way failure and setbacks can lead us to mastery.

So what is grit? It’s having a thick skin in the face of defeat. Grit ties together ideas we’ve already discussed, like surrendering to pain and striving for mastery.

Taking a step back, it’s important to note that grit is different from persistence or self-control. Persistence is rugged steadfastness towards an end goal – like studying hard to pass your medical school exams and become a doctor. Self-control is more temporary – like resisting temptations.

By contrast, grit is endurance displayed over years. It’s about continuing to strive for mastery despite apparent failure, over and over again.

For example, the director of Iowa University’s world-famous Writing Program noted that the most successful writers were those who most wanted to become great – not the ones with the most natural talent. So in the case of the most ambitious writers, grit is a matter of continuous, unrelenting effort over time.

But grit isn’t only about making a continuous effort, it’s also about applying it towards mastery. And that’s why art is one of the best ways to learn grit. After all, art teaches us to constantly reassess ourselves, our work and our ideas – even as we continue to strive.

Consider that artists have to deal with criticism all the time. Well, successful artists have figured out how to use valuable parts of the criticism to improve their work, and discard the rest.

Imagine standing next to a painting you poured your soul into, while dozens of critics figuratively tear it apart. How would you respond?

In the end, having the grit to face that kind of criticism is what you truly need to succeed. So find a way to absorb all the criticism, pain and difficulties you encounter, and incorporate them.

Failure, as we know it, doesn’t really exist. When we confront what looks like failure, instead of bowing to it, we need to see it as it really is: a valuable learning experience. Because after all, if you want to accomplish great things, you have to find a way to transform your setbacks into motivation. In other words, you have to continually strive for more in order to achieve more.

Action plan: Next time you think you’re at rock bottom, tell yourself: “Great, I can only go up from here.” After all, although being at rock bottom might feel like a failure, it isn’t! It’s part of the path towards achieving your goals.