AI Can't Be Accountable, So We Must Be

This Tom's Hardware piece, and the timeline of Jer Crane (CEO of PocketOS) correctly place accountability for the recent destruction of his company's production data with Cursor, Claude, and Railway, but do not include Jer Crane himself. The phrase a computer can never be held accountable is even truer today than it was when first written in an internal IBM training manual back in 1979 because of how much more capable computer systems are today. Crane's timeline is valuable whether you're an engineer building systems that leverage AI, an executive selecting dependencies, or an engineering manager between them. This piece will explore those details to model a post-mortem with full accountability.

Crane lays out the stakes of his software working as-designed clearly:
"We build software that rental businesses — primarily car rental operators — use to run their entire operations: reservations, payments, customer management, vehicle tracking, the works. Some of our customers are five-year subscribers who literally cannot operate their businesses without us."
A single API call destroyed business-critical data for every PocketOS customer in 9 seconds. Railway's architecture storing volume-level backups in the same volume meant all the backups of that data were also deleted. They experienced what every business that uses AWS us-east-1 experiences when that region has an outage--but worse, because current backups no longer existed.

Crane includes the agent's "confession" in his post-mortem. Reading it in full reiterates numerous ways in which people granting an AI system both a broad scope of action and control over its own guardrails violates the principle of separation of duties with catastrophic results. Talking about Cursor's safeguards failing or the difference between Cursor's marketing and the reality of the tool in action doesn't change the fact that "a computer can never be held accountable". Buying access to Claude's most expensive model and configuring it "with explicit safety rules in our project configuration" integrated with Cursor does not transfer his share of accountability to Cursor, Anthropic, or Railway. Where specifically did the post-mortem fall short?

Railway's documentation on data & storage backups has a Caveats section which describes backups as "a newer feature that is still under development". Given this, a post-mortem that talked about migrating to managed database outside of Railway to enable multi-region database failover as a way to mitigate the risk of that feature would highlight the platform shortcoming and demonstrate a CEO taking proactive steps to make his service offering more resilient.

Per Crane's timeline, the agent encountered a credential mismatch while working on a routine task in a staging environment. The post-mortem doesn't explain what this routine task was, or why a credential mismatch led to volume deletion rather than a failed operation and a logged error. The lack of additional detail on this is an accountability gap for the PocketOS team, and ultimately the CEO. The same tool being used to make staging environment changes, generate code on its own, help engineers develop code, review it, and deploy it constitutes a separation of duties (SoD)failure. Before GenAI, my prior company used a GitHub bot to require different engineers to review and approve PRs for merge than those who implemented the change. If a team failed to install that bot on their repos, any SoD violations detected by the CI/CD pipelines would block production deployment until the violation was resolved or an accountable executive agreed to take responsibility for any production issues resulting from that deployment. Multi-layer SoD enforcement mechanisms are still needed even with GenAI.

From Railway's API documentation, the token created "to add and remove custom domains via the Railway CLI" was an Account token--the only type with "all resources and workspaces" scope. A post-mortem which pointed out the different available scopes and replacing that token with a Workspace or Project token scoped to staging would create a balanced picture of the CEO being accountable and reinforce his good points regarding the absence of feedback and human-in-the-loop opportunities in the token creation flow. Railway's token types are not as granular as those offered by classic personal access tokens or fine-grained tokens in GitHub, but they do provide the limited scope which limits the blast radius of misuse (accidental or intentional). The API tokens aren't time-limited, so a long-term step the CEO might recommend in the post-mortem is a Railway feature providing an expiration option as a security measure customers can choose to use. Crane's post-mortem does a lot of things right--but the missing piece is himself. Cursor, Anthropic, and Railway need to be accountable--but so does he.

Regardless of the field of endeavor, we need to take accountability for the outcomes of actions taken by generative AI systems. Here's what that looks like in practice:

  • Build guardrails that people control, instead of delegating that responsibility to AI.
  • Apply the principle of separation of duties anywhere AI agents act independently.
  • Use environment-scoped access so that credentials which work in non-production environments don't work in production environments (and vice versa).
  • Govern AI agents' scope of action by the principle of least privilege--just as we should for people.
  • Require generative AI to ask permission for every change, even in your personal use--this shifts accountability back to people.

Companies Trying to Replace Middle Managers with GenAI Will Pay a Price

Organizational (re)design choices by tech companies are changing the role of software engineering leader in ways that make the job harder--and generative AI won't fill the gaps. Flatter organizations deprioritize developing people, which ultimately harms project delivery and undermines strategy as well. This piece by Meg Adams on the necessity of a shopkeeper mentality for software engineering leaders not only provides an effective leadership model but a description of the things we can do in the role which generative AI can't.  

We need to do what we actually say our job is instead of spending our time in ways an executive without proximity to the work might think generative AI can replace.  "Scanning” our work domain, adapting to the reality of today’s hybrid and distributed workplaces is how we can build, refine, and maintain the context necessary to deliver projects, drive strategy, and develop people.  People can delegate—but maintain ownership and accountability. AI can't. We are uniquely capable of scanning--but also of micromanaging.

Flatter organizations can't know what they need to about how their people are doing, or where the friction is, or what they haven’t been seeing because they’ve eliminated the managers who could tell them and they lack the systems or signals that could tell them. Friction in key communication channels within the organization and blockers to work progress may eventually show up in other ways, but perhaps not before missed deadlines or costly mitigations occur.  Why?  Because the remaining middle managers in a flattened organizational structure have too wide a span of control to proactively identify and address issues that pose obstacles to work progress. In addition to the visibility issues, flatter organizations eliminate growth paths for employees. What does professional growth look like in an organization where you’re either an individual contributor or the CEO with little or nothing in between except generative AI? Where will the next senior leaders come from?

Believing generative AI will provide credible answers to these key questions seems exceedingly optimistic.  What we’ve seen so far of how the efforts by companies trying to replace entry-level workers with AI does not inspire confidence.  Senior engineers in these companies are burning out from overwork and leaving to find employers that enable them to better balance their personal and professional lives.  Senior engineering leaders who face the prospect of directly managing dozens of people may make similar decisions rather than suffer burnout. Companies betting that generative AI can fill the gap are risking their long-term viability.


The Hidden Cost of "Just Build It with AI"

Someone recently posed this question in a Slack channel at work: “why pay for vended software that we can build ourselves now that we have the tools?”  They asked this question in the context of Claude Code being made generally available for internal use.  It's a fair question on the surface. These tools can be expensive, sometimes frustrating, and the capabilities of modern LLMs are genuinely impressive. But the growing power of Gen AI doesn't fundamentally change the buy-versus-build decision — here’s why.

 

Vibe coding is not a production strategy.

Yes, you can prototype a Salesforce-like CRM or a Slack-like messaging tool in a weekend with the right AI assistant. But if your business is going to depend on it, you're not just building a demo. You're committing to building, deploying, and maintaining something production-grade — with all the architectural decisions, security concerns, and UI considerations that entails. GenAI may accelerate parts of that process—but it doesn't eliminate any of them.

 

Now do the math on the opportunity cost.

Building your own internal replacement for a third-party SaaS product isn't a one-engineer side project. It's multiple engineers, sustained over time, making real decisions about infrastructure, reliability, and scale. That means:

  • Salaries and benefits for the team dedicated to this effort
  • Switching costs to migrate off whatever you're currently using
  • Ongoing maintenance — because software doesn't maintain itself

 

And critically: every hour those engineers spend building your proprietary version of Slack or ServiceNow is an hour they're not spending on the software that is actually core to your business.  What advances are your competitors making at your expense because they’ve fully dedicated their people to improving their core business functionality and bought tools that are tangential but helpful?

 

That's the real question to ask.

Is replacing a third-party tool — one that entire companies exist to build and support — actually a better use of your engineering talent than investing in the capabilities that differentiate your product in the market?

For most companies, in most cases, the answer is no. Gen AI is a powerful accelerant. But it doesn't change the fundamental logic of what's worth building versus what's worth buying.


The Price of Replacing Entry-Level Workers with AI is Too High

 

A friend shared this Fast Company piece in our group Slack recently on the costs to companies of replacing entry-level workers with GenAI.  Issac, the mid-level Big Tech engineer in Megan Carnegie’s piece, is like a  lot of engineers working on my teams.  He’s seen firsthand what GenAI accelerates in his own work, as well as the downsides of its incorrect outputs and the additional time necessary to fix them.  At my level, I must balance the increased demands for software delivery, a larger people leadership workload, the risks of burnout for my most-experienced engineers, and the downstream consequences of that burnout.

 

Contrary to the assumption that GenAI could make up the difference for fewer junior engineers, much more work rolled up to senior engineers.  Instead of growing junior developers to the point they can do more work autonomously, they’re herding AI agents and spending an extra 4.5 hours a week just fixing their mistakes—in addition to doing their own work.  When you have a dozen or two dozen direct reports, it’s much more difficult to detect and prevent burnout.

 

When a burned-out senior engineer leaves, you don’t just lose their knowledge and experience.  The tacit knowledge built through what they worked on with those in their teams and across teams is lost as well.  GenAI appears to be compounding the negative effects we’ve seen of repeated mass layoffs by numerous companies—loss of institutional knowledge at scale.  The vicious cycle of burned-out senior engineers being replaced by less-experienced engineers and smaller workforces overall is already leading to less-resilient systems with more frequent outages taking longer and longer to resolve.

 

As a people leader, replacing entry-level workers with GenAI also eliminates opportunities for me to delegate, mentor, and teach.  The vicious cycle has the paradoxical effective of devaluing me as a person who has grown institutional knowledge over the years and consistently shared it with others to help them grow professionally.  At least one study has shown that this mentoring work increases motivation and psychological well-being.  I’ve worked with dozens of engineers over the past 8 1/2 years in peer learning, people leadership, and mentoring contexts and contributed to at least 10 people getting promoted at least one level in that time.  More than any individual system I’ve led delivery of, helping engineers grow their careers to succeed at even greater levels feels great.

 

These costs of trying to replace entry-level workers with AI are only going to grow as experienced workers who can afford to retire choose to do so, creating a demographic cliff Carnegie’s piece measures at 4.6M more workers retiring than there will be equivalently-qualified younger workers to replace them.  Companies that keep hollowing out their future talent pipelines are placing their continued viability at risk.  Those that use GenAI to augment  their existing workforces instead will have a more sustainable future. 


Software Development Roles: Lead versus Manager, Revisited

I first wrote about the lead vs manager distinction in software development in 2016. A decade later the line is blurrier than ever--and GenAI is part of why.

The original distinction was simple: if you spend half or more of your time writing code, you're a lead, regardless of what's on your business card. If significantly less, you're a manager. Organizations that keep those roles separate consistently ship better software and grow better engineers.

That's still true today. But GenAI has given companies a new reason to collapse these roles--and a new set of risks when they do.

Here's what's changed for each role:

For the Tech Lead:
GenAI raises the productivity floor for individual contributors dramatically. A skilled lead with strong AI tooling can produce what used to require a small team. Companies have noticed--and some are using this to justify asking leads to absorb management responsibilities. Don't be fooled by the productivity argument. Reviewing AI-generated code at scale, governing how your team uses these tools, and maintaining architectural integrity are already expanding the lead's cognitive load. Adding people leadership on top of that isn't a productivity gain. It's a recipe for the same burnout the hybrid role produced before GenAI existed.

For the Engineering Manager:
GenAI doesn't eliminate the core of the management role--it shifts it. The administrative overhead (status updates, reporting, some forms of performance documentation) should get easier. What doesn't go away: removing blockers, developing people, judgment calls about team direction, and now--understanding AI-assisted output well enough to hold your team accountable for it. AI-assisted--not generated--because people need to remain accountable for any and all software that ships. No blaming the AI for outcomes. Managers who stay deliberately close to how their teams are using AI will have a real edge over those who don't.

The job-search advice from a decade ago still holds: scrutinize the description, ask how the role is scoped, and look at the manager's span of control. What you find will tell you whether the company has thought clearly about these roles — or is using a familiar title to describe an unfamiliar burden.


Navigating the Great Delayering, Revisited

It’s been less than a year since I wrote this piece about engineering leadership inside a delayered organization.  Here’s what I’ve learned since then.

When your rollup goes from 7 to 14 to 21 (or more) direct reports in short order, something has to give.  What gives first is actually developing the people who report to you.

Delayering isn’t just a management inconvenience.  It breaks the leadership pipeline.

In a healthy organization, people leadership starts small—a handful of reports, room to make mistakes—and learn from them, space to coach and be coached.  Delayering eliminates that entirely.  In my last organization, people leadership now starts with responsibility for two engineering teams.  There is no shallow end of the pool anymore.

The way my last organization implemented delayering, team leads were turned into individual contributors with little warning beyond rumors on Blind.  Imagine the whiplash of being promoted to lead a team in January only to be converted to a lead IC just a few months later.  Nearly a year after implementation, there are still no clear growth paths or guidance to the next level for dozens of people in my line of business alone who went from being team leads with responsibilities for helping their direct reports shape their careers to individual contributors. 

The ripple effects of delayering are real:

* 1:1 time shrinks.  Feedback gets less tactically useful because the manager is too far from the work.

* aspiring leaders never get to test whether leadership is for them before being thrown into the deep end.

* senior ICs have to absorb mentoring responsibilities that aren’t formally recognized or rewarded.

* existing leaders get stretched until they break—and the institutional knowledge walks out with them.

The companies that leaned first and hardest into delayering (Meta in 2023, Amazon from 2024 to the present) have not obviously improved their products.  In Amazon’s case, the combination of delayering, layoffs, and ramped-up GenAI usage has led directly to more frequent and severe production incidents.  The already-fleeting margin improvements of delayering may yet turn into losses as a result.

When I evaluate my next role, among my first questions will be whether the professional growth of direct reports is a genuine leadership priority—not a talking point.


How I Created My First LinkedIn Carousel Post with Claude and Canva

A friend recommended I post more original content on LinkedIn as I enter the job market this year.  Here’s how I used Claude, Canva, and other tools to create my first LinkedIn carousel post.  First, I prompted Claude to look at my blog archive and curate the best 20 posts with career advice.  Next, I asked Claude to convert one of the recommended posts into a LinkedIn carousel. After manually editing the HTML to update the content for the present, I opened up Canva and added screenshots of the HTML slides to a LinkedIn carousel template.  After downloading the carousel in PDF format, I attached it as a document to a post and shared it with my network.

I’ve been blogging off-and-on for over 20 years, not always about my career, but enough that I expected there would be enough posts I could use as a starting point to share original content on a consistent basis.  Here’s an outline of my initial Claude prompt that could be adapted and applied to any LLM:

<blog archive URL>
Review the provided URL and generate a table with the top <number of posts> links best suited to repurposed as LinkedIn posts.  Each row should have the following columns: link to the blog post, <number of> items of key career advice the post contains, recommendation on whether significant rewriting is needed to make it relevant to the present day.

While I didn’t provide a persona in my prompt, I did provide details on how the response should be structured.  Here is some of Claude’s “thinking” as it built a response to my prompt:

Curated relevant career posts and prioritized content extraction
 
Curated relevant career posts and prioritized content extraction
 
 
 

 

 
 
favicon
 

 

 
 
 
 
 

 

 

  1.  
  2.  
  3.  
  4.  
  5.  
  6.  
  7.  
  8.  
  9.  
  10.  
  11.  
  12.  

 

 

 
 
 
 
 
 
 

Now let me fetch the key career and leadership posts to understand their content before building the table.

Curated twenty career posts and assessed their modernization requirements
 
Curated twenty career posts and assessed their modernization requirements
 
favicon
 

 

 
 
 
 
 

 

 
 
favicon
 

 

 
 
 
 
 

 

 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
 
 

 

 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
 
 

 

 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
favicon
 

 

 
 
 
 
 

 

 

 

  1.  
  2.  
  3.  
  4.  
  5.  
  6.  
  7.  
  8.  
  9.  
  10.  
  11.  
  12.  
  13.  
  14.  
  15.  
  16.  
  17.  
  18.  
  19.  
  20.  

 

 

 

 

 
 
 
 
 
 
 

I now have enough content from the key posts to build a comprehensive table. Let me compile this into a document.

Curated twenty career-focused posts with relevance assessments and update recommendations
 
Curated twenty career-focused posts with relevance assessments and update recommendations
 
 
 
 
 
 
 
 
 
 

 

 

 

 

 

 

 

 

 

 

 

 
 
 
 
 
 
 

Now I have enough content to build the full table. Let me create a nicely formatted HTML file with the table.

Created a file, read a file
 
Created a file, read a file
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Here's your LinkedIn repurposing table with all 20 posts ranked and analyzed. A few highlights worth calling out:

Ready to post with minimal editing (5 posts): "Navigating the Great Delayering," "Tell Me About Yourself — Engineering Leader Edition," "Software Development Roles: Lead vs. Manager," "Idolatry of Innovators," and "The Problem with Exit Interviews" are all essentially evergreen and could go up this week with only length trimming.

...

Format tip: Several of these (especially #2, #11, and #13) would convert naturally into LinkedIn carousels — the structured advice translates well to a slide-per-point format.

 

Claude’s response was a nicely-formatted HTML table.  The column with blog post links included the date of the blog post.  The “rewriting needed?” column provided unexpected depth regarding the degree of rewriting needed, recommendations of specific updates to be made, even advice on the best time of year to share a particular post.  The format tip which came just before the HTML table is what sent me down a bit of a rabbit hole to try and create a LinkedIn carousel instead of a traditional post. 

Building on the output of the first prompt, and advice I read elsewhere regarding LinkedIn carousels, my next prompt was this:

As a LinkedIn content specialist, turn the contents of post #2 into a LinkedIn carousel of 5-15 slides in length.

Unlike my initial prompt, I did provide a persona.  The range of slides gave Claude enought flexibility to size the content to a sensible number rather than stretching or cramming it into too many or too few slides.

The HTML version of the carousel looks great—I’ll definitely make future carousels I publish available in that format as well as in LinkedIn.  The main goal was posting on LinkedIn though, so I pressed forward.  On my Mac mini, I used Cmd-Shift-4 to screenshot each HTML slide as a image.  Then I dragged and dropped it into the template pages and resized the images to fit.    After downloading as a PDF (a few times as my final walkthrough of the output revealed some things I wanted to tweak), I uploaded and published the post to my LinkedIn network.  In the future, I’ll make the posts open beyond my LinkedIn network for broader reach.