Featured

If you’re totally comfortable to launch, you’ve waited too long.

It’s both exciting and nerve-wracking to develop highly visible production service platforms. Exciting, in that the ideation phase of creating a differentiated offering inspires “creatives” in the software engineering space to do their best work. Nerve-wracking, in that knowing the details of implementation, and what hit the editing room floor during planning/prioritization exercises, carries with it a certain sense of responsibility — especially if you end up being caught-out on any of those areas as “real usage” begins.

There was a fantastic paper written last year on the topic of creativity in software engineering. Highly encouraged reading, especially for software engineering managers, technical team leads, those involved in hiring/promotions and so forth.

In order to solve today’s complex problems in the world of software development, technical knowledge is no longer enough. Previous studies investigating and identifying nontechnical skills of software engineers show that creative skills also play an important role in tackling difficult problems.

Exploring the Role of Creativity in Software Engineering, Jan 2021, https://arxiv.org/pdf/2101.00837.pdf

For this blog, I’ll focus on the experience the team I led went through during the bring-up of a highly visible service launch scheduled for March, 2021 called Red Hat OpenShift on AWS (ROSA).

The ROSA service was a first-of-its-kind, first-party offering (transacted through Amazon) that would be added to the AWS console. That set the high bar for the team to deliver on what the business partnership between Red Hat and AWS had negotiated. Shortly thereafter additional first party services from partners were added to AWS console (Grafana has one at least). But indeed ROSA was the first 🙂

That led us within engineering to struggle a bit with not only quantifying readiness, but also a gut-check on whether we’d launch with a sufficiently differentiated offering that was compelling enough to drive sales and delight customers. Lessons from lean management, lean startup and so forth are salient here, and were appreciated and emphasized by both management and technical leaders who are responsible for our engineering culture.

I estimate that if you did a straw poll at the time about our readiness, the room would have been split 50/50 on a go/no-go. My impression on that split was that we all as humans have different “risk defaults” that play out more generally in our life (think about modulating risk in one’s investment portfolio). The leadership team’s feedback was “If you’re totally comfortable to launch, you’ve waited too long.” And that’s totally right. While we have very high standards not only for ourselves as engineers and SRE, but also for Red Hat as a whole in terms of reputational risk, getting out there into the market, and building the the habit of “going to the Genba” is the correct and primary path to success.

What it’s like to be “how constrained” and becoming a free thinker

Featured

I think it’s most folks (engineers especially) default approach, when being introduced to a new system, product or idea, to immediately want to understand “how” something works. This blog is to specifically challenge that default, turn it around, and pose that we default to “why” something works. Why it has to exist, and why it is the way it is.

People are “how-constrained”. I have this thing, I know how it works. And then little tweaks to that will generate “something”. As opposed to: what do I actually want, and then figure out how to build it. It’s a very different mindset. And almost nobody has it, obviously.

Jim Keller & Elon Musk

The flaw, maybe fatal even, about being “how-constrained” is that you have a tendency to ignore, or not fully appreciate, the bigger picture found in “why”. Let’s take for example building a product at your company.

By the time requirements hit engineering, most of the “why” has been discovered, measured, discussed, justified and “talked out”. I argue that if product and management teams do not clearly articulate, strategically and in “addressable-market” terms, the “why” to engineers, you miss an important cultural motivating factor in building healthy teams. You also miss an opportunity to have some of the brightest minds check your work. Never skip the step of clearly articulating “why” we’re embarking on building a new thing, and how decisions were taken.

That is not to say that the position of “naturally worrying about how” is not a critical behavior and attribute. It is that we must continually question our “why”. There must be a feedback loop, and one of the expected outcomes of that is that status quo’s are continually challenged.

Lastly, I’ve noticed that being “how-constrained” inherently limits vision in even the most talented groups. I’ve also noticed a soft correlation between seniority and being less how-constrained, although that seems to be valid only at the most senior levels. If you see yourself over-rotated towards “how”, try for a moment being “why-constrained” and see how it frees your thinking + gets your creative juices flowing. I know that it does for me.

Lessons learned while bootstrapping new $things

During my career at Red Hat, I’ve had the great pleasure of being a go-to person to bootstrap $things. After a good amount of repetitions, I’ve arrived at a bit of a blueprint on how to do it. Each go at it becoming more and more refined and incorporating lessons learned as each $thing presents its own nuances.

As an engineer exploring ways to bootstrap $things, Simon Sinek’s book Start with Why deeply resonated with me. I began looking for frameworks that let me capture “why” in a way that has unyielding precision, and found the Patrick Lencioni’s Six Critical Questions section in his book The Advantage.

I’ve now used this format several times and am continually impressed by how durable it is. If you can capture the essence of the new $thing in this format, you can look back on it even years in the future, and your intent remains unambiguous.

Further, and most importantly, executive stakeholders and beyond can see exactly what’s being bootstrapped, the why and how. This leaves no ambiguity — all that remains is execution and follow-through.

From Patrick Lencioni: “Creating alignment at the executive level is essential to building and maintaining a healthy organization. There is probably no greater frustration for employees than having to navigate the politics and confusion caused by leaders who are misaligned. Even the slightest bit of daylight between executive team members can cause an overwhelming effect on employees below. There are six simple but critical questions that need to be answered, eliminating all discrepancies among team members.”

This approach reduces the risk of that daylight ever seeing the light of day.

Most recently I bootstrapped the internal special interest group for Site Reliability Engineering called SIG-SRE. I even made a fun video for the kickoff 🙂

Here are the (lightly edited for this blog) Six Critical Questions for clarity + answers for that project:

  • Q1: Why does SIG-SRE exist?
    • The SIG-SRE project is an effort to seize horizontal opportunities created by Red Hat offering managed services.  SIG-SRE will support efforts like Operate First, internal communities of practice, and Red Hat University.
  • Q2:  How do we behave?
    • We develop consensus amongst accountable parties and output process and procedure to be implemented amongst teams.  We will extract strategic value from in-practice SRE teams and bring reusable value back to ourselves.
  • Q3:  What do we do?
    • We will lead in building the practice of SRE across the company.  We will bear down on the core shared problems that enable Red Hat to build a competitive managed services business.
  • Q4:  How will we succeed?
    • We will take an upstream first (Operate First) approach to ensure Red Hat’s core ethos is carried forward into the managed services space. You can see this commitment delivered here.
      • The vast majority of participants will participate in on-call rotations for their existing service teams.
      • We will be accountable for goals that the SIG participants set, guided by the needs of groups and the company at large.
    • Use the Open Decision Framework
    • Ensure new processes include clear roles and responsibilities (RACI)
    • Ensure the SIG is deeply tied into training efforts to enable existing associates to succeed (e.g. PMs, traditional developers)
  • Q5:  What is most important, right now?
    • Build consensus about the opportunity and approach.
    • That the SIG becomes a force multiplier, allowing engineering teams to benefit from economies of scale, lifting all our boats.
  • Q6: Who must do what?
    • If this problem space resonates with you, discuss with your management chain and tech leads how your team can best support SIG-SRE by allocating specific engineering and BU/PM resources.  We will not achieve escape velocity with favors and part timers.  The value to a team in allocating engineering time to the SIG should come back to them, in multiples, in a measurable way, in a finite time frame.