Hero's Journey

1 Summary

I think of a software project is a Hero’s Journey for the people on the team. They gain skills, find mentors, confront their fears and overcome adversity. When the project is released they are changed by what they’ve learned.

The Hero’s Journey is Joseph Cambell’s Monomyth. It lays out the elements of the Hero’s Journey:

1 Summary
2 The 17 Stages of the Monomyth

2.0 The 17 Stages of the Software Monomyth

I will try to map the above stages to their equivalent in a software engineering project.

2.1 Separation

This is usually starting a new job, or a new project or role within an existing job. The key is that you leave the familiar terrain of your former, more comfortable job. If you’ve never had a job, separation is the point at which you get your first job.

2.1.1 The Call to Adventure

The essence of an adventure is to cross from a position of ignorance to mastery. All software projects are all an exploration into the unknown. It is intrinsically difficult to create an unthinking automaton that behaves correctly in all predictable circumstances. It is also intrinsically difficult to understand exactly what someone else means or wants when they describe their view of a large system. Do you remember the story of the Blind Men and the Elephant?

The call to adventure isn’t supernatural: it is sometimes just an email. Or, your impending journey might simply be mentioned in passing in a meeting.

“Oh, by the way, now that your UVW work is done we’re thinking of putting you on the XYZ project.”

All good stories have an antagonist, and this one does too: fear, ignorance and chaos. Make no mistake: these are formidable enemies and underestimating them can doom a project to mediocrity or utter ruin.

2.1.2 Refusal of the Call

Everyone has their comfort zone and sometimes we are called to learn things that are outside of it. I will admit I have not been willing to learn certain technologies because I simply do not enjoy working with them. That has hurt me more than once in my career, but I have also been helped by focusing on the technologies I do appreciate. I think it is ideal to have skills in the entire software stack, including system administration and hardware repair; however, it takes a long time to learn that much and in the mean time, we find ourselves on projects that require us to step outside our technology comfort zone.

Do you take the challenge? If you don’t, hopefully it is because there is a more enticing set of adventures waiting for you. Follow your passion, always. But, different technologies are like different lands: you can expect a different culture and level of sophistication or rigor. If it isn’t outside your career path, take the journey there and back again.

2.1.3 Supernatural Aid

What is supernatural? What is magic? I used to feel sad because I could not find magic in the world. There were no magic items, rings, wands or spells. No amount of thinking could move a cup across the table remotely, I found. I felt sad that I could read about “magic” but was forced to live in a mundane world.

But, then I realized that meaning is magic.

Data isn’t meaning. A book is full of printed words, but the story doesn’t spring to life without an observer who can bring meaning to that data. An observer requires a mind. Ours is manifested by a brain with hundreds of billions of simultaneously operating neurons. Meaning meets my definition of magic: I can ask someone at the table to hand me the cup and, it usually works.

If meaning is magic, software is magic.

So in the context of “software is magic” supernatural aid comes in a few forms:
* Open-source systems that are already close to or a superset of what is needed for the project
* Pre-existing libraries that solve some very hard part of the problem (such as encryption, optical character recognition or voice recognition, etc)
* Previous versions of the project that worked
* Similar projects to which you have access or can remember
* People who have a wealth of experience in the technologies you’ll be learning or developing
* Process and techniques that break complex problems into tractable ones
* Knowing what not to try

Supernatural aid is aid beyond the universe of your own mind and what you can directly experience. It is magical and you know it when someone explains why the approach you’ve been struggling with for days actually cannot work and that the best practice is to use another approach which you later discover is easy to adopt. It’s magic because before someone told you that you would have struggled until you failed. With a few words they changed the way you think about the problem and you changed what you did as a result. You were changed by the process of getting around that opportunity for failure: ideally you won’t ever fail that way again.

When you start a new project, the first step is to learn about what it is you are supposed to accomplish. If it isn’t documented and you can’t glean it from what is documented, write the documentation yourself. You’ll end up knowing it better and everyone after you will have the benefit of reading what you wrote. It is how organizations gain knowledge instead of just individuals. If someone isn’t willing to be part of helping their organization gain knowledge, they are acting only in their own short-term self-interest. The organization will be weaker for it, and that has a longer-term affect on all its members.

2.1.4 Crossing the Threshold

If you accepted the project then sooner rather than later you will find yourself either in a blank editor wondering how to begin, or looking at an existing code base and wondering what it all means. Or, you might be trying to make an estimate for creating something you don’t understand yet.

One way or another you take a step into the unknown when you begin to dive into the project. You might begin by reading the relevant standard, or studying example code, etc., but however you dive in you are simultaneously at the nadir of your understanding and the furthest from the end goal. If you understood this little just in front of your deadline it would be terrifying: like facing a dragon with a wooden sword. But, mercifully, the deadline is still far off and as it approaches our ignorance fades. The project comes into focus as time goes on by the end it is in perfect focus and all the parts are finished. Ambiguity has been excised from the shared vision of the product.

But, taking that first step is like stepping through the looking glass: the world on the other side is almost necessarily mysterious until we have explored and discovered and learned.

For a while all you can do is take in the strange landscape and try to stay conscious.

2.1.5 Belly of the Whale

If the project were a train ride, then crossing the threshold would be stepping onto the train. You are in the Belly of the Whale when the train leaves the station and you can no longer get off comfortably. I don’t really like the whole Biblical Whale metaphor...

I prefer to think of it as the point at which you enter your own natural discovery/design/implementation/test process to take on each thing that needs to be done. It’s chemistry in the belly of the whale, and plenty of projects have a stomach ache, or an ulcer. Projects where there is not a clear, shared vision get sick over time. It’s almost inevitable if there isn’t a solid, shared vision. I personally define leadership as the part of someone’s behavior that establishes a shared vision among a team. Any team member can provide leadership. What matters most is that a shared vision is created that really is shared among the team and interested stakeholders. Anyone can be a leader, but not everyone is willing to let others lead.

The Belly of the Whale is where the software engineering process begins to operate.

2.2 Initiation

This is your experience in the project: every unknown you confront and eventually overcame by learning something new.

The thing is: you can do this. If others can learn it, so can you. You’ve probably proven this to yourself over and over, so keep it in mind. There probably isn’t technology you can’t learn if you have access to clear, relevant information about it. This is also why if you support the idea of a fair meritocracy in the workplace you’ll make an effort to provide clear, relevant information for other people about your own work. You’re the only one who can determine if your work can be taken over by others.

So, write some documentation if it’s missing. Someone wrote some for you, so at least return the favor to your community.

And yet, sometimes people feel fear about new technologies or problems in a project. Just remember: fear is strongest in the dark and can’t survive in the light. You bring the light with you in the form of all your experience and knowledge and techniques and cleverness and intelligence. The new technologies and problems should fear you.

Just bring your willingness to find out. There are surprisingly few monsters under the bed.

2.2.1 The Road of Trials

Each task in a project is its own initiation steps. This is not by any means a complete list:
* Learning new Technology or Language
* Learning a new Problem Domain
* Learning a new data structure
* Learning a new API
* Documenting knowledge for oneself, one’s team and the organization
* Designing and Implementing a new feature and making it work in all typical cases
* Interfacing with real data sources
* Upgrading for Branding or Localization
* Abstracting features to make them data-driven or configurable
* Adding SQA measures passing all tests
* Releasing into production
* Maintaining production
* Managing cross-version compatibility or upgrades
* Upgrading to the next version of the product
* Integrations with compatible systems
* Additional Reporting
* Understanding the politics of stakeholders

Really, your project could be as simple as a web clipboard (simply storing one string per user) and it would still go through many phases that could each involve confronting the unknown.

And, since most projects are actually quite complicated compared to the hypothetical clipboard project, this gauntlet of software development maturity awaits team members on dozens or hundreds of separate tasks. Each task has its corresponding portion of the above initiation steps.

The adventure likely requires confronting a thousand fears, one at a time. By the time you succeed, fear will your victim.

2.2.2 The Meeting with the Goddess

Joseph Cambell’s definition of this step makes it pretty clear that this isn’t related to women or even a female spirit. And, I’ll add that this isn’t Deus ex Machina either.

Campbell writes:

This is the point when the person experiences a love that has the power and significance of the all-powerful, all encompassing, unconditional love...

I think this is the moment when you can see how it will all fit together and suddenly you cannot type fast enough. I think of this moment as an epiphany that can last for days or weeks. It feels as though all troubles and worries are finally lifted from your shoulders and you can see the way to succeed. You finally get it.

I see a system that works as a thing of beauty. Understanding how it works is being able to appreciate more of that beauty, especially in ways that could not be appreciated without that understanding. Filesystems are easy to use and we might think of them as simple things. They are, until you try to create one and realize how difficult it is to make a reliable one.

For me, the feelings that come with understanding a project are part of why I am a software engineer. Order can be alluring. Creating a great deal of order makes me feel powerful. So, I am driven to get to that point where I truly understand the project. You could say that I am not satisfied until I have met with the Goddess, but I’d prefer to say that until I understand all the important parts I am not yet at my best and not yet ready to finish the project.

2.2.3 Woman as Temptress

This is about temptation, not women or sex. In a software project there are many temptations. It is tempting to:
* Do what’s safe and easy
* Do what has been done before
* Not raise difficult questions
* Not look past the next release
* Focus on only one’s own part and now how it fits within the system
* Use inferior technologies because you already know them
* Make assumptions without checking them with others they affect
* Implement abstractions or automation that doesn’t have a good return on investment of time, because they are more interesting or actionable than the main project
* Raise doubts and argue instead of trying something new

The motivations of an individual in a software project are driven by their values. Some people want recognition. Some want variety. Some people want safety. I think these typically overlap with success; however, sometimes projects fail and everyone tries not to be perceived as the cause. That can lead to some recriminations and finger-pointing, especially in cases where there was not a solid and clear shared vision of what was supposed to be done.

Without a clear shared vision, the entire team is responsible for failing to recognize that they weren’t all clear on what was being built and for failing to build that shared vision together.

I find it disappointing that the software industry has a problem including women. And I also find it disappointing that temptation has been characterized this way, with “Woman as Temptress”. Sure, there is a historical, sexist basis for this. But phrases like, “Woman as Temptress” don’t help. I think I’d call this step ‘Stay on the Path’.

Stay on the Path

In my experience, the closer you get to your goal, the better are the temptations to step off the path. If it was a road, you can imagine more and more tempting turn offs. Success requires focus. However, if you are clever you can remember where these opportunities are and come back to them as soon as your main goals are met. They aren’t always there when you return. But, if you don’t turn aside then the only possible destination is success if you don’t give up. If you do turn aside to temptations, you may never reach your original goal.

Of course if you had all the time you wanted you could take every detour. But, you don’t. So, each detour represents an incremental risk to finishing at all.

Failure only happens when you quit. Temptations are one way Chaos shows you where you are in your journey. You probably think of Chaos only as a foe: it isn’t always. It is just an honest representation of order in your system. When Chaos is throwing minutia at you, you’re in a different place than when it’s throwing business plans at you, or when it throws unexpected scaling or demand because what you made is useful enough to attract more users.

2.2.4 Atonement with the Father

This is about finding the ultimate balance between cost, duration and quality. They say, ‘fast, cheap and high quality: pick two’. They say that because if you try to specify all three it isn’t clear the project can be completed at all. Whatever isn’t handled by any two of the trio ends up in the third. So, fast, cheap software has quality issues. High quality software delivered quickly is very expensive. And, cheap, high quality software, if such a thing exists, takes a long time to manifest.

You could argue that open source software is cheap and high quality and doesn’t take long to manifest because it already exists. You’d be right! If your software project can easily be replaced by open source software, perhaps the project should not be run at all and instead, the open source software should be used instead. But, if one can’t take advantage of free, open source software, then the fact that it exists is irrelevant, and Atonement is a big issue. Because, even the head of the biggest software companies cannot violate the truth of cost, duration and quality.

Atonement has a connotation of not being someone’s first or comfortable choice. It is about finding out deeper truths whether one imagined they existed or not. If a software project is run poorly you can bet there are plenty of lessons to be learned. If the project comes to completion, the final answers about cost, duration and quality can be known after-the-fact. Hindsight can make fools of us all.

2.2.5 Apotheosis

When the project is finally released there is often a flurry of post-release activity, or even another release right on the heels of the first. But, near the end of the release cycle the engineers start spoiling for the release by making changes sparingly. They change only what is needed to fix any important problems before the release is made. Sometimes problems are sanctioned and the team agrees they will not be fixed in the current release.

This is a good time to go back and update documentation. I often find problems in my pre-release work simply by documenting it. It is also a good time to reflect on the project to understand what were the main problems and main successes.

If the shared vision led the way for development, this phase revises the shared vision so that it corresponds to what ended up being made. This is the beginning of the end of the journey, and it may mark the end of new work on the project too. It might feel as though something is missing that used to be there on the project. It can feel powerless, as though one is a ghost, because the project is frozen and more changes are not allowed. Whereas before you could use your knowledge and skill to change and evolve the project, now you may not, except for minor tweaks here and there.

You aren’t dead, but it is possible your involvement with the project nearly is. You might feel sad or mortal. This is one reason I like to work on more than one project at the same time, so that I am never exclusively in this mode.

2.2.6 The Ultimate Boon

I think the ultimate boon is the knowledge one gains about how things work. It isn’t the software produced. It isn’t the sales made or dollars you are paid. It isn’t the market share captured. It isn’t even the relief of finishing the project. Those things are all important, of course. Perhaps you would never even be on a project without them. But, I think the ultimate boon is what you take with you to the next project. It includes:
  • Your experience
  • What you learned to do
  • What you learned not to do
  • What you learned from what didn’t work
  • What you learned about how to solve problems
  • What you learned about yourself
  • The confidence you gained
  • The fears you lost

It would be tempting to place material gains within the bounds of the ultimate boon. Perhaps I am revealing my heritage that I value what I learn more than what I own; however, I think there are no magic swords, only magic words. Meaning is magic. Words are quantized meaning. When you learn you increase your store of meaning. You increase your supply of magic. More magic means the world is a correspondingly less scary and dangerous place. No amount of knowledge can eliminate all fear and all risk. However, our feeling of fear is driven by our ignorance and learning undermine’s fear’s supply of it, making it correspondingly less powerful.

The results of receiving ultimate boon is the reduction or even removal of fear. This is a worthy goal: it is a terrible fate to live in fear.

The journey of a single project leaves you with more knowledge and understanding. So even if the project fails you will still receive your ultimate boon. But, it might taste like failure. Whether it tastes of material success or not, if you grew through the process it could at worst be a material failure.

Do not reject or trivialize the ultimate boon you receive from working on a project. There truly are diamond on your path and all you have to do is pick them up to own them. You do not need to leave your path for these treasures. You are supposed to pick them up as you stay on the path.

When you get home you can ponder their nature and appreciate them better (and look for elvish runes).

2.3 Return

Return to where? To your career. Yes, a successful job is certainly part of your career (if you work hard and are lucky); however, your career is not your list of employers. Your career is what you’ve learned and become as a person. It is the sum total of your experiences and all you’ve learned. So, when you linger in a job and aren’t learning much anymore, your career has stalled. If you aren’t learning you are most certainly falling behind. The cyber world evolves so quickly that it is very easy to get left behind. If left behind for too long you may find it very difficult to catch up again.

2.3.1 Refusal of Return

Security and impunity are narcotics. They can so seduce a person that they may stay in a project long after all the diamonds of learning have been picked up. It’s comfortable. It’s safe. It’s easy. It’s the course of least resistance, especially if the project is successful and achieving its material goals. It’s tempting to stay in such a project, sometimes long after you’ve ceased to learn anything new. But, you know something is wrong. Despite all the success, you’re bored. You hunger for new things to learn, as if it were a food you couldn’t go without. The hunger gnaws at you until consciously or not you force yourself away from the familiar project and into a new one. It’s better if this is a conscious process.

Before the change from the familiar to the new you were in a special place: you knew enough within the project to be productive, creative, even heroic. You were powerful. You had magic on your side and you probably vanquished bugs as if they were, well, bugs. Indeed, your vanquishing skills would have swollen to a point where your name might even become a verb within your organization. Or, managers might compete for your time. It can feel disorienting to have people meet for an hour about how to spend fifteen minutes of your time. It bathes one’s ego in warm flowing reassurance and any ideas about being an impostor in your own job fade away.

And so, with such temptation and security, it is easy to refuse to return to your career.

2.3.2 The Magic Flight

What you know and have learned on the project is not at risk of be taken from you. However, knowledge and learning are not sufficient: what is learned must also be right.

It’s easy to draw the wrong conclusions, to mismatch cause and effect, to confuse correlation with causation and to misunderstand anything that could be understood. So, your bag of ultimate boon may be filled with rocks if you are not careful about what you put in it. Not all shiny stones are diamonds. Not even all hard, clear gem-like substances are diamonds. Your path is paved in stones and some of them are clear, so you must be careful when you stoop to pick one up.

You are in control of what you take away from any experience. The facts of the situation might be beyond your control; however, what you learn and infer from those facts is up to you. You are both free to and responsible for making the most of it. You must determine if they are a diamond or glass or chewing gum or a rock. That is the Magic Flight: your struggle to take away knowledge and learning and not misinformation. It isn’t always that easy to separate valuable learning from lessons that would actually be better not to learn. This is doubly true on projects that fail or struggle. It can be easy to learn the wrong things in a project that fails; however, you need not be a victim.

The Magic Flight isn’t difficult because the Gods are angry that you’ve stolen knowledge. The Magic Flight is difficult because Chaos is the natural state of matter and information and if you want something more valuable than chaos out of your experience you will have to put some effort into creating some order in your learning. At the same time, our natural condition as humans is to learn. We probably could not stop learning if we tried. Somewhere between “everything is Chaos unless you work to make it otherwise” and “you can’t stop learning if you tried” is where we really live: We learn things but aren’t certain how correct they are.

Only future projects can test what we’ve learned and reveal whether it was a diamond that cuts through any problem or a rock that destroys what it hits, or chewing gum that just gums things up and makes them work poorly, or glass which cuts until it shatters and leaves you hunting for a real diamond and some band-aids.

The Magic Flight isn’t difficult because of opposition. It is difficult because facts can appear in a confusing or misleading order. It is difficult because more than one thing can be broken at the same time leading to confusing cause-and-effect conundrums. It is difficult because we don’t have perfect access to information or visibility into all parts of a program at the same time. It is difficult because there’s more than one way to look at things, and because we all have our own habitual thinking patterns that naturally bias us toward or away from some kinds of knowledge. It is difficult because of our limitations in learning and growing.

It is difficult, but we’re building ourselves and we’re worth the effort.

2.3.3 Rescue from Without

Unless you work on a project by yourself, sooner or later one or more other team members will probably rescue you. Rescue you from what? Making a big mistake or wasting a lot of time or being unable to fix a problem.

Before the project is finished, all the important bugs will be fixed. All the important features will be implemented. Before the project is finished all its parts will cooperate and exchange data as they were designed.

Unless you work on a project by yourself, this means your work and the work of other team members depend upon each other. A sender and a receiver must both be working for a message to be received. While it may be true that the problem lies with the sender or the receiver or both, it is also true that the message is not being received and that the ramifications of the failure can transcend the sender or the receiver software. The failure of the message to arrive means something in the real world. Maybe the failure is unimportant and the request will be made again. Or, maybe it was an email, telling you that you were hired, and the fact that it wasn’t delivered to you made you think you were passed over.

Unless you work on a project by yourself sooner or later you and other team members will get together to make sure your parts work together. At first they won’t, but later they will so long as you all shared a common vision of what was to be accomplished in the project.

Since you have been focusing on your part of the project, you probably do not know as much about the other parts of the project, at least, not to the same depth that you know your part or the other team members know theirs.

When two big parts aren’t working together, you have a choice:
  1. Figure out what is wrong on your side or their side and fix it yourself
  2. Engage their help with their side and each of you fix your own side together

The problem with #1 is that it might take you anywhere from a few hours to a few years to acquire the background knowledge and experience to determine what the problem is on their side, and the appropriate way to fix it. Almost everyone jumps to #2, and this is the route through which rescue from without occurs.

The project cannot be finished until all the parts work together. It may very well be that the problem is inside of your own part of the project, but you can’t determine what it is and how to fix it without the help of someone else on the team. Let me repeat those very important words:

It may very well be that the problem is inside of your own part, but you can’t determine what it is and how to fix it without the help of someone else on the team.

It may not be possible to fix your part without their help. Not. Possible. Utter failure awaits without their help! This is a rescue operation. You’ll probably feel grateful when they give you the information you need to fix the problem in your part of the project. Hopefully this leaves you more willing to help them when they confront a problem in their part that requires your part to illuminate.

I’ve developed quite a few client-server systems, and this is always true: we are each other heroes.

If you work on a project by yourself, you must be your own hero or you must seek help from outside. But, if you wrote it, nobody else knows how it works.

I’ve never felt so lonely in my life as when it seemed there was no person on the planet who could answer my question about how chaos had crept into my software. But, my approach is to try to explain how the software works to someone else or even myself (out loud), and often times one of the outcomes of this exercise is a list of new things to consider or try, or even an insight into the actual problem and fix. Even pretending that another person is there to listen to you explain your code can lead to a solution: this is virtual rescue from without. The mere notion of an outside rescuer is sometimes sufficient.

2.3.4 The Crossing of the Return Threshold

Knowledge we’ve gained cannot be taken from us without destroying us. And the Magic Flight hopefully allowed us to sift through our experiences for the real gems of learning. The Return Threshold is moving on to another project, even if the next project is simply the next version of the current project. But, at some point we must move on to the next project by crossing the threshold to leave the previous project.

This is natural in engineering organizations when one release ends and a new one begins. But, it can be a chaotic time. Will the next version be developed at all? Will they start over with a new team and technology approach? Will you take this opportunity to find a different project or a different job? It is a time of significant change. Compared to the daily routine of project meetings and development, this time lacks structure and even a definite timeframe.

You might be ready for the next project before it is ready for you. What will you do in the mean time? And, what if you’re assigned nothing to do and later people wonder how you spent your time?

2.3.5 Master of Two Worlds

Once you’ve moved on from a project it enters your corral of tamed solutions. The solutions you keep in that corral are there for you to bring out and use whenever you wish. You are the master of these steeds and they allow you to do great deeds. You can rig them together to solve even bigger problems.

You are better for your participation in the project, even if it was uncomfortable or the project failed or struggled, and even if you personally failed or struggled. In fact, more so if you failed or struggled. Failure is the greatest of teachers.

If the project succeeded and you grew and learned, it came at a cost: some of your fears. You will have to give up some of your fears with each new project you finish. Some of them simply cannot survive in you anymore. You’ve achieved a spiritual victory over fear and ignorance. We are changed by it. Over time we achieve a better balance between the material world and the spiritual world, as the Monomyth suggests, precisely because we shed our fears and become ready and willing to embrace new concepts more easily.

2.3.6 Freedom to Live

When we are fearless we are the best engineer we can be.

When we are ready to embrace any new technology,or problem domain and have the confidence and experience to know we can succeed, we are free to work on the widest set of problems and to find solutions for them. That means the largest number of jobs are waiting for us and the risk of being out of work is minimized. This is survival. This is the equivalent of being better at finding food, or evading predators, or choosing when to migrate, or when to invest in education instead of ritual.

Fear is part of our human experience, and we each have a different tolerance and ability to recognize it within us. However, each software project is a chapter in a life-long conflict with fear.

You simply cannot lose unless you quit.