52 Aces

Learning, competition and capitalism

Complexity

Volatility, Stability & Contradictions

To maintain an accurate or effective grasp of reality one must undergo a continuous cycle of interaction with the environment geared to assessing its constant changes.

If you’re reading this, then you’re interested in the concept of volatility. Furthermore, I’d wager that you’re here because you want to find some sort of solution to the “problem” of volatility in your own life.

I don’t blame you for feeling this way. Every passing day seems to generate more complexity, rapid change and uncertainty than at any other point in human history.

You aren’t wrong — that’s exactly what’s happening. But below the surface of this chaotic ocean of volatility lurks useful knowledge.

Knowledge that will make you uncomfortable in many cases. Knowledge that will make you realize you’ve only taken a few steps down the rabbit hole.

But it will be knowledge nonetheless. If you’re the sort of exploratory person who enjoys endless journeys and moments of insight that make you feel like you’ve just unplugged yourself from one reality and jerked your nervous system into a new one…well, you’re in the right place, my friend.

Volatility

The world we live in is intensely volatile. You know I don’t need to go down the standard laundry list of examples, because you’re living in it. Every day it feels like a new order is being shaped in one domain, while the old ways and traditions are somehow holding back progress in others.

Once again, you’re not wrong — this is exactly what’s happening.

Here’s the issue with such high levels of volatility: you weren’t designed to cope with it. Small bites of volatility are fine, you and every other human comes equipped with enough adaptive capacity to deal with some change.

We know how to respond to unexpected weather, changes in social relations, patterns of behavior in our rivals, and so on. These were all part of the deal for early humans who possessed the same brain you possess at this very moment.

What you and I were not designed to cope with were global pandemics, flash crashes, the overnight disappearance of long-standing institutions, and all the other seemingly insane changes that occur in our globalized world.

Defining Volatility

You might also be wondering what I mean when I say volatility. This is a fair line of inquiry, especially since the only people who purposefully use the word volatility on a regular basis are traders (usually options traders, but traders as a group tend to be interested in it).

For this crowd, volatility carries a specific definition (courtesy of Investopedia):

A statistical measure of the dispersion of returns for a given security or market index.

In more straightforward language: how much the price of a security deviates from some expected average price.

But what if we want to boil volatility down to its essence, right on down to the fundamental level so that it can be applied to domains beyond finance?

You’re luck! We can, in fact, do such a thing by restating the definition of volatility in more general terms:

Volatility is a high rate of change within a given environment.

Notice that this definition is free from value judgements — we’re not saying that volatility is either good or bad, it just is. We’ve reframed it from a phenomenon with specific characteristics, to a sort of measurement.

All you’re saying when you say “volatile” is “high rate of change” — nothing more, nothing less.

How You Feel

Volatility, like any other conceptual model, doesn’t care about how change makes you feel; such a concept is based entirely on our own subjective experience of volatility.

The reason I bring this up is important to focus on: most people speak about volatility in strictly negative terms. You can see it whenever the term “volatility” pops up in either the media or casual conversation.

A couple of quick examples highlights what I mean:

  • “Greg has a volatile personality” is not a compliment. What is implied by the use of “volatile” in this context is that Greg is unreliable, untrustworthy and unpredictable. And we hate people who fall into those buckets.
  • “The markets have been volatile” is, for most people with money in the markets, not a good thing. The average investor wants straightforward, predictable returns. Volatility ensures that uncertainty about those returns rules the day.

Stability

What is it that we want instead? Stability, aka safety, aka comfort.

We can define this complimentary concept of stability within our new framework of change measurement:

Stability is low rate of change in a given environment.

Stability makes us feel all warm and tingly on the inside. When we know what to expect and have removed uncertainty from a situation, we breathe easier.

It’s a natural human instinct. Knowing what comes next feeds into our intrinsic need to predict the future — and stability in an environment gives us that.

Much like volatility, the idea of stability has many affective properties built into our descriptions of it. Let’s see how stability makes our previous examples look:

  • “Greg has a stable personality” is definitely a compliment. Greg’s the kind of guy you know will show up when he says he will, will complete the tasks he’s been assigned, and is a generally trustworthy guy.
  • “The markets have been stable” is a positive for most investors. Stability (or at least our concept of it) implies a healthy, dependable economy that produces high, predictable returns.

It’s clear that we treat the idea of stability as a net positive nearly every time we utilize it. Stability translates to predictability, and predictability is what we’re all programmed to seek. Therefore stability is viewed as in intrinsically positive concept.

Contradictions

The contradictions that we generate as people are what I’m most interested in and what I spend much of my time reading and writing about. If you’re trying to understand the world you live in, you should do the same.

Contradictions are everywhere, and recognizing where they pop up is a critical skill you must develop — especially if you want to understand volatility.

This is where things start to get interesting, and now we take our first steps into the maze of complexity. Here it will start to become clear to you just how deep the rabbit hole goes.

For one, both high and low levels of volatility can be good or bad, sometimes both simultaneously.

Consider the subtexts of these statements:

  • “Our marriage is stable.”
  • “I have a stable career.”
  • “My schedule is stable.”

What’s your first reaction when you hear someone say these?

While the surface context is positive, it’s not difficult to dig up the contradictory downsides within each one:

  • A “stable” marriage can mean that two people get along very well, and it’s clear that they’ll be together for the rest of their lives. On the flip side, it can mean that they never speak to one another or have given up trying to work out difficult problems that have been festering under the surface for years.
  • A “stable” career can mean that your trajectory up the corporate ladder is assured and there is nothing but blue sky and stock options ahead. It can also mean that you’ve been sidelined into the same role for a decade, or that you’ve gone as far up the ladder as you can get and still aren’t where you want to be.
  • A “stable” schedule can mean you’ve discovered your ideal lifestyle and have the means to live it on a daily basis. It might also mean that you’ve been locked into a machine-like routine that you both hate and can’t escape.

Shooting Yourself in the Foot

The contradiction here is that when you’re always grasping for stability, you often shoot yourself in the foot and end up with more volatility.

Even though a stable environment sounds great on paper, in practice a lack of change usually provides results opposite to what you were looking for.

You could, in theory, achieve perfect predictability by locking yourself in a dark room and never leaving it. A prison represents a high degree of stability, as prisoners have mandatory routines that are defined for them.

But you have to ask yourself: what benefits come from such low levels of volatility? Is creating a prison (either mental or physical) really the best move? Does a zero-volatility lifestyle really sound so appealing if it means doing nothing?

For a certain type of personality, I suspect the answer is “yes.” The idea of being “institutionalized” by prison life is well-established, and many people leave prison-like conditions only to find themselves feeling lost as a result.

If that’s you, then you need to listen to what I have to say more than anyone else. Why? Because in a world ruled by volatility, an intolerance for change is a death sentence.

Stability requires us to make changes. This is the core of the contradiction of stability: our constant search for stability leads us into volatility.

Why do you think we call careers “the rat race”? We’re all scurrying around, desperate to latch on to stability in the form of job security, high pay checks, and a predictable path up the ladder.

Changes

But how do we accomplish this? By making changes, constant changes. We try out different clothes, take online classes, read books, join clubs — we can’t help but make dramatic shifts on a regular basis, hoping to finally hit that road to the paradise of stability.

Maybe someday, through all your racing through the maze, you’ll be able to reach that peak of zero-volatility: paid retirement, a carefree lifestyle and margaritas on an exotic beach. Then you’ll finally get what you want…when you’re old, and have spent the best years of your life drowning in volatility.

At the end of the day, you never escape volatility in a meaningful way. Change is a constant, no matter how hard you try to escape it.

What about somebody who always seeks volatility? A similar contradictory result shows up much of the time here, too.

I remember watching a show about prison a long time ago, and one of the prisoners described the reason he was always heading back to the big house in a way that I still think about:

“I just live life by the ‘balls-to-the-wall’ philosophy.”

In other words, he was always looking for excitement and that quest for peak experiences always took him down the path of prison time.

This sums up perfectly what happens to many people who live for a constant rush. Their quest for volatility at all times leads them to what they dread the most: indefinite stability.

The ultimate example is someone who puts themselves in harm’s way all the time and ends up dead.

What is death, if not the purest form of stability?

Synthesis

We’ve established at this point that there are two forces at play (stability and volatility), and that there’s a central contradiction between the two of them: seeking one out exclusively tends to give you the opposite of what you want.

Pure stability-seeking leads to high levels of volatility, and pure volatility-seeking leads to high levels of stability.

What is a fundamental, first-principles idea that can be gleaned from all of this? What is it about volatility and stability that burns deep into the very fabric of existence?

In short: what is the synthesis that bursts forth from everything you’ve just read?

This is my answer:

Volatility is a fundamental concept because it emerges from the contradictory relationships built into the complex systems and narratives we create.

Phrased differently, volatility is the unavoidable consequence of:

  • The systems we create, which end up being complex because of how interconnected everything is now.
  • The narratives we construct to explain those systems, because there is inevitably information asymmetry. The asymmetries exist between those who created the systems, those within the systems, and those who are outside of the systems.

This is why volatility is so critical to understand. Everything around us is complex, and everything around us has some narrative surrounding it. This means there will be contradictions, and in turn volatility will be present.

It’s a complete mind-fuck of an idea: we must expose ourselves to high rates of change, and we must be ready for the explanations we find to contradict each other. Conflict is a constant, there is no escape.

From all of this, it becomes clear that volatility isn’t just some concept for traders to toss around as a way to describe deviations from expected prices. It’s far more fundamental and important than that, and you should turn it into a central tenet of your thinking.

Some Semblance of Balance

Think of the relationship between volatility and stability as being like an inverted U shape. In the middle of the inverted U is the ideal mix of volatility and stability, but too much stability or volatility (either side of the inverted U) is counter-productive:

The sweet spot between volatility and stability.

By seeking out this “sweet spot,” we can at least approach the contradictions of stability and volatility in a constructive way. Instead of the living death of pure stability, or the untenable chaos of pure volatility, we can probe the world for situations where different amounts of each are appropriate.

My goal is to live somewhere near this balancing point between stability and volatility. Throughout the years, I’ve found that this is a difficult task — sometimes it’s impossible.

In some ways, the concept of equilibrium falls into the same bucket as volatility: a contradiction that forces you away from your goals. If all you ever seek is equilibrium, you’ll be disappointed in how infrequently you find it.

Such is the nature of exploring complex topics. Sometimes (most of the time), there aren’t neat, easy solutions.

In many cases, you’ll find that ideas you hold sacred need to be completely rearranged, or thrown out all together. In some cases, you might get lucky and realize you were on the right track.

Either way, be ready for a long trek.

In the meantime, here are some guidelines that I use for thinking through the problems volatility presents:

  • Embrace the fact that neither volatility or stability possess affective properties. They aren’t good or bad, they just are. Whether they have a positive or negative impact depends on the environment and the people involved.
  • Utilize the Thesis, Antithesis, Synthesis approach when looking at complex problems. TAS can be summarized as finding a concept, coupling it with a competing concept and then finding where the two antagonisms intersect. Utilizing this method will help you identify the contradictions within any given domain, recognize the crossroads between what people mean and what people practice, and see where solutions might lie.
  • Do your best to embrace volatility and not spend so much time looking for stability. Consider the inverted U and ensure you have a mix of volatility and stability in your life. Pure volatility and pure stability are both poisonous, so as long as you aren’t too far in either direction you should be alright..

How to Talk About Algorithms

This is part of The Algorithmic Society, an ongoing series about how algorithms and algorithmic thinking have taken over the world. Want to know when new content shows up? Sign up for my newsletter here.

This invasion of one’s mind by ready-made phrases can only be prevented if one is constantly on guard against them, and every such phrase anaesthetizes a portion of one’s brain. 

-George Orwell, Politics and the English Language

The term “algorithm” has become the latest linguistic tool for sounding sophisticated when talking about technology, a sort of TED Talk-esque shortcut to identifying with the Silicon Valley set. There’s something about the word itself that mystifies the average mind and imbues the user with an air of sophistication.

When someone at a cocktail party starts using the world “algorithm,” it becomes evident that this person is well-read and keeping up with the times. In other words, it’s become another way to signal status by leveraging (the appearance) of technical knowledge.

The word itself has become a stand-in for the god-like power that the largest internet platforms hold over our daily lives, a device for describing the black boxes behind behemoths like Google and Facebook. How we use the term “algorithm” hints at a sort of cowed awe at the sheer magnitude of their impact on the modern world.

We don’t know how they operate or who really pulls the levers (I imagine someone thinking are there levers on algorithms? as they read this) behind the curtain. Owners of algorithms are fine with this, because it makes their lives easier — they get to keep trade secrets to themselves, and they’re given a veneer of respectability in the process.

Ian Bogost described this dynamic best:

The next time you hear someone talking about algorithms, replace the term with ‘God’ and ask yourself if the meaning changes.

-Ian Bogost

Part of this has to do with the connection between algorithms and the gargantuan, public fortunes they’ve created in the era of high technology. There’s a new sort of American dream associated with algorithms, most famously captured in the movie The Social Network.

There’s a scene in that film where Eduardo Saverin and Mark Zuckerberg (played by Andrew Garfield and Jesse Eisenberg, respectively) are working on an algorithm by writing on a window.

Eduardo Saverin (Andrew Garfield) works on an algorithm with Mark Zuckerberg (Jesse Eisenberg)
Eduardo Saverin (Andrew Garfield) works on an algorithm with Mark Zuckerberg (Jesse Eisenberg)

While the algorithm in this scene is for a hacked-together pet project, the implication of the context is clear: algorithms with humble beginnings can conquer the world. Multiple billion-dollar fortunes were spawned by this primordial bit of mathematics that got translated into code.

Now algorithms are all over our cultural landscape, which is itself dominated by business narratives and the investors who drive them.

For example, it is now unimaginable to create a startup that didn’t offer some kind of algorithm that’s at least a minor improvement over another, existing algorithm. Not only is it not cool, but venture capitalists tend to shy away from such businesses because they lack scale. 

In short, algorithms are ever-present in our social media, on our phones, and in every domain imaginable that involves what could be considered “technology.” Algorithms are everywhere, algorithms are in everything, algorithms are everything.

Marc Andreesen famously said “software is eating the world,” but I would argue that the more correct phrasing is “algorithms have already eaten the world.”

Defining Algorithms

An algorithm in action
Image source

There’s an odd contradiction at the heart of our views on algorithms. On the one hand, we all seem to understand (to varying degrees) that algorithms play an outsized role in our lives. On the other, nobody seems to know what the word “algorithm” means.

What’s even weirder about this is that this definitional problem isn’t confined to the tech-illiterate: even computer scientists don’t have a generally-accepted definition of what an algorithm is.

It’s a long-standing debate, and some even say it’s not possible to sort algorithms from non-algorithms because of a computer science concept known as the halting problem.

This ambiguity can be viewed as a positive or a negative, depending on the algorithm and how you relate to it. But that’s all part of a larger exploration that I’ll get into later. For now, we need to come up with some kind of starting point for discussing algorithms that makes sense.

Since there isn’t a single, unified definition of an algorithm, we’ll have to use a simplified (and therefore flawed) one for now:

A set of well-defined steps for solving a specific class of problems.

What’s nice about this definition is that it gives us the ability to generalize the idea of an algorithm beyond its mathematical and computational origins. We can use it to describe any process that’s designed to operate in a repeatable, predictable manner.

We could say, for example, that low-tech activities like cooking involve the use of algorithms. After all, a recipe is a set of unambiguous steps (add ½ cup sugar, bake for 25 minutes, etc.) for “solving” specific food-related problems (how to convert ingredients into food, which in turn solves the problem of being hungry, etc.).

Bureaucratic procedures at a large company or government organization are also algorithmic if we run with this definition.

An employee can also be seen as a sort of algorithm as well — their whole job is to perform a specific set of tasks in order to accomplish goals for the organization. They do this by performing algorithms for each problem they’re presented with.

A Transformative Force

It’s also worth adding another dimension to this definition:

Algorithms transform some set of input values into a desired output or set of outputs.

This is key to understanding all thing algorithmic. Algorithms transform what they take in and generate something novel in the process.

In some sense, this is the most important way to look at algorithms. It makes you realize that there’s some objective involved, that what the algorithms are creating isn’t just math — they’re machines of creation.

Algorithms aren’t just transforming inputs from computer systems, either. As they integrate more and more with physical objects (including people), they are using the real world itself as a set of inputs and creating new landscapes in their wake.

Now we can start to unravel the linguistic consequences of using the word “algorithm” the way we do. By talking about algorithms as our digital overlords — never to be questioned or examined in a meaningful way — we hand them power they don’t deserve.

Even though they’re designed by high-caliber computer scientists, these algorithms are still designed and operated by humans. This means they’re flawed in countless ways, and even our most powerful computers can’t rid themselves of their designer’s human errors.

When you embrace that fact, it becomes clear that talking about algorithms with a glint of admiration in our eyes is often a mistake. While some are worthy of praise, quite a few are far more fragile, inaccurate and exploitable than their owners would like you to know.

More than anything, we need to get rid of this idea that algorithms are simply hand-waving mechanisms for explaining new technology. Algorithms are real, they serve specific purposes and it’s possible to at least begin to understand how they operate if you equip yourself to probe them.

Their impact, despite their flawed nature, is large. We’ve built, and continue to build, an algorithmic society, and it’s simply irresponsible to treat algorithms in such a haphazard way. It is the duty of every intelligent, capable adult in this new world to get a handle on what algorithms are and how they are shaping our world.

And this starts by learning to talk about algorithms not as magical code-driven dragons. It starts by seeing how they’re infiltrating not just our computers, but our very identities, our everyday existence.

They are fractal, spinning themselves through increasing levels of abstraction as they generate billions of dollars and shift our personal lives in ways that even their creators often don’t understand.

Algorithms are, in short, the most important topic of the modern era. It is my goal with this series to give you a glimpse into just how large of an impact they’re having, and then provide you with the tools you need to navigate this world more intelligently.

There will be more to learn, but consider this the starting point.

The Exploration-Exploitation Dilemma, Simplified

This is part of my 5 Minute Concepts series, which is designed to help you understand fundamental concepts about subjects like learning, memory and competition in the shortest time possible. Each episode is available in video format on my YouTube channel and audio via my podcast. If you prefer to read, the transcript is below.

Want to know when new content shows up? Sign up for my newsletter here.

Transcript:

I’ve written about the exploration-exploitation dilemma before, but only in a long-form essay format. Since I think this is such a critical concept, and I realize that not everyone has the time to read a big essay, I’ve created this simplified explanation.

Just a warning: like any other 5 Minute Concepts piece, there’s always more to the story. I’m just trying to give you the most important parts in 5 minutes or less.

Anyway…

Let’s start with a stripped-down definition: the exploration-exploitation dilemma is the choice we all have to make between learning more or taking action with the knowledge we already possess.

Learning more is exploration, acting with current knowledge is exploitation.

With either action you’re trying to find some way to maximize what’s often referred to as “reward,” or some end-state that you find desirable.

The reason this is a dilemma is simple: you can’t explore or exploit exclusively and win in the long run.

If all you do is explore, you’ll never take action in the world — which means you get a predictable payoff of exactly zero. There isn’t much to be gained from passively gathering information until you die.

On the other hand, taking action without learning anything is also a long-term losing strategy. You do get some kind of reward by exploiting a known path, but that means you’re giving up any chance at a higher payoff that might be staring you in the face without you knowing about it.

The real kicker here is that exploring is what drives the value of exploitation, and vice versa. You need to explore in order to find good paths for exploitation, and you need to exploit in order to get a reward for your exploration. Both actions are dependent on each other.

What you’re balancing in either case is opportunity cost. You have a limited amount of resources, such as time and money, to work with over the course of your life. If you explore, you’re by default not exploiting, and vice versa.

Consider this example: Let’s say you’re scrolling through Netflix, looking for something to watch for the new couple of hours.

You notice that a movie you’ve seen a dozen times is one of the choices and consider watching it. Right next to that is a movie you’ve never seen before.

Choosing the movie you’ve seen provides a specific emotional payoff for you. You know all the best parts and you’re well aware of how the entire experience will make you feel.

Choosing the movie you’ve never seen means taking a certain amount of risk. There’s an unknown payoff for watching this new movie, and it might end up being a waste of two hours. Those two hours will be gone, never to return.

But you might also discover a new favorite movie, or genre, or director, that you never knew about.

It’s easy to get sucked into either extreme. I’ve known people who spent their whole lives reading, accumulating a veritable library worth of knowledge in their head, but never tried to do anything with it.

And, of course, I’m sure we both know people who have never read a book or stopped to think for even a moment about whether their beliefs and actions should be altered in some way.

While this is an unsolved problem (and trust me, many people have tried to figure it out), there are some good rules of thumb to run with. First of all, don’t favor a binary approach. Only exploring or only exploiting doesn’t work in the long run.

Secondly, it pays to spend a lot of time exploring early on and then shifting more and more to exploitation over time. But — this is critical — you never stop exploring completely. For a person in the real world, exploring should always be part of your strategy.

There’s always some accommodation made for learning new things. This is known as the epsilon-decreasing algorithm, and, if you just want a simple heuristic for managing this dilemma, it’s a pretty good place to start.

Third, there are always inflection points where it makes sense to shift from one to the other. Sometimes it’s a moment where you realize you’ve finally reached a level of knowledge that grants you a new level of competence and the time to utilize it has come. Passing a professional exam is a simple example of this.

Other times you might suffer a bitter defeat and receive an unfiltered signal that it’s time to explore. If a big project you’ve been working on fails, for example, you might need to go back to the drawing board and evaluate how to improve for your next attempt.

I could talk about this for hours, but in general I want you to understand this: figuring out how to spread your time between exploration and exploitation is perhaps the most important problem you’ll ever face.

Don’t push this into the background — be conscious and deliberate about it. Doing that might just change your life in ways you never saw coming.

The Illusions of Memory

Let’s do a little exercise. At this very moment, I want you to stop doing anything else you’re trying to do aside from reading this article. Stop checking your phone, close other windows, turn off any music or podcasts you’re listening to and just focus on this.

It’s tough to do sometimes, but I promise you it will be worth it.

[Read more…] about The Illusions of Memory