IDP: Beyond Self-Service to True Developer Agency

In the world of platform engineering, the push for an Internal Developer Platform (IDP) is often framed as a clear journey from chaos to order. The goal is to boost developer productivity and create a seamless self-service experience. The story begins in a state of nature, a chaotic landscape of bespoke shell scripts, half-forgotten Jenkins jobs, and tribal knowledge whispered between senior engineers. To deploy a new service, you have to find the right person, who knows the right incantation, to run the right script on the right server. It’s slow, brittle, and terrifying. So a team is formed, the Platform Team, tasked with taming this wilderness. Their charter is to build a "paved road," a standardized, automated path from code to production. They will replace the patchwork of scripts with a unified API, a slick web UI, and a promise: self-service. No more tickets, no more waiting. Developers will be able to provision infrastructure, deploy applications, and manage their services all on their own. The platform will set them free. This is a noble and compelling vision. It’s also, in most cases, a mirage.

The Self-Service Mirage
The promised land of self-service rarely materializes in the way we imagine. What often emerges instead is not a tool for liberation, but a new and more sophisticated form of bureaucracy. The human gatekeepers are replaced by automated ones, and the ad-hoc constraints of the old world are codified into the rigid, unyielding logic of the new platform.

The problem lies in our framing of the goal. We aim for "self-service," but what we end up building is a vending machine. Developers can select from a pre-approved menu of options - this size database, that type of cache, this deployment pipeline and the machine dispenses it. This is certainly an improvement over chaos. It's predictable, it's repeatable, and it's fast, as long as what you need is on the menu. But what happens when it's not? This is the moment the illusion of self-service shatters. A developer needs a slightly different version of PostgreSQL, or wants to add a specific sidecar to their service mesh, or requires a network policy that the platform's UI doesn't support. They hit a wall. The "self-service" platform tells them "no." Their only recourse is to go back to the old way: filing a ticket with the Platform Team. They are no longer self-sufficient; they are once again supplicants, waiting for the high priests of the platform to grant their request. The platform, designed to eliminate bottlenecks, has become the single greatest bottleneck of all. We've traded a flexible, high-friction system for a rigid, low-friction one. We've optimized for the 80% case, the common denominator, but in doing so, we've made the other 20% the cases involving innovation, experimentation, and unique requirements nearly impossible. The paved road is wonderful, until you need to go somewhere it doesn't lead. Then it's just a barrier.

The Real Goal: Not Service, But Agency

Perhaps the goal isn't "self-service" at all. That term implies a transactional relationship, one where the developer is a customer to be served. But developers aren't customers in that sense. They are creators. What they truly want isn't to be served, but to have agency. Agency is the power to act, to make meaningful decisions and see them through. It’s the feeling of being in control of your own destiny, of having the tools and the freedom to solve your own problems. The old world of chaotic scripts, for all its faults, offered a perverse kind of agency. If you were clever enough and brave enough, you could bend the system to your will. You could write your own script, SSH into the box, and make it do what you wanted. It was dangerous and unscalable, but it was powerful. A successful platform shouldn't destroy that agency; it should channel it. It should make it safe, repeatable, and available to everyone, not just the wizards.

This suggests a different model for a platform. Not a vending machine that dispenses finished products, but a well-stocked workshop.

The Platform as a Workshop

A workshop doesn't give you a finished birdhouse. It gives you a reliable power saw, a sturdy drill press, and a set of sharp chisels. It provides well-maintained, powerful primitives, and trusts you, the crafts-person, to use them to build what you need.

What are the primitives of a developer platform? They are things like: a secure way to get cloud credentials, a system for defining an application and its dependencies, a mechanism for observing its health, a way to store secrets. The job of the Platform Team is not to build a monolithic UI that hides these things away. It's to provide them as robust, well-documented, and composable building blocks. Instead of a UI with a dropdown for "database size," provide a stable, versioned Terraform module for provisioning a database. The module can have sensible defaults that cover the 80% case, making it easy to do the right thing. But it should also expose the underlying parameters for the developer who needs to venture into the 20% case. The platform's value isn't in hiding the complexity of AWS or GCP; it's in curating it. It's in providing expert-level opinions as sane defaults, while always leaving an escape hatch for the expert user. This reframing changes the role of the Platform Team. They are no longer just software engineers building a product. They become something closer to product managers for the internal developer experience. Their primary metric of success is not the number of features in their UI, but the reduction of cognitive load for developers. They must be obsessed with the developer's workflow. What are the points of friction? Where do they get stuck? What takes them out of the flow of writing code?

A New Contract for Platform Engineering

A great platform team understands that their job is to build leverage, not control. They provide a foundation of reliability, security, and observability that every developer can stand on. The contract becomes: "You focus on your application's business logic, and we will provide you with a foundation that makes it easy to run that logic securely, reliably, and at scale. We'll handle the undifferentiated heavy lifting of IAM policies, VPC networking, and Kubernetes upgrades."

The "self-service" part of this new contract isn't about a web form. It's about the developer's ability to interact with this foundation programmatically and on their own terms, using the tools and primitives the platform provides. It’s the ability to define their infrastructure as code, right next to their application code, using modules maintained by the platform experts. It's the ability to get observability data for their new service automatically, without filing a ticket to configure Prometheus.

This approach also solves the "paved road" problem. The platform provides a paved road made of these primitives, and it's the best and easiest way to travel. But if a developer truly needs to go off-road, they are not met with a wall. They are met with the same building blocks the platform team used to build the road itself. They can take those primitives - the credential system, the application definition and compose them in a new way to solve their unique problem. The platform doesn't prevent this; it enables it, albeit with the understanding that if you go off-road, you're taking on more responsibility for your own support. You have the agency to make that choice. When we look back at the initial chaos of scripts and tribal knowledge, we can see the problem wasn't the existence of the scripts themselves. The problem was that they were unowned, unmaintained, and undiscoverable. They were liabilities. A platform built on primitives turns those liabilities into assets. The Terraform module for a database isn't a one-off script; it's a versioned, documented, and supported piece of software, with the Platform Team as its maintainer. It captures the organization's best practices, but in a living, usable form.

The journey from tooling to self-service is a misleading narrative. It suggests a destination where the Platform Team has automated itself out of a job. The real journey is from chaos to curated agency. It's a continuous process of building trust and reducing friction. It’s less about building a single, monolithic "platform" and more about fostering an ecosystem of well-maintained tools, modules, and libraries. The ultimate success of an IDP isn't when everyone is using its fancy UI, but when developers can build, deploy, and operate their software so smoothly that they almost forget the platform is even there. It just feels like the natural way of working.

This reframing from a product to a curated ecosystem is a subtle but profound shift. How does your organization balance the paved road with the need for developer agency?