I’ve been thinking a lot about the future of consulting lately. This is a bit of an occupational hazard… I’ve been a consultant for almost 20 years, based in San Francisco, and I’ve run my own practice for eight years and counting.

A long time ago I remember telling a colleague:

“The consulting space is massive: multi-billions of dollars globally. Even if it were to shrink by 50%, there’d still be money on the table for those who know how to find it.”

That remark, though not particularly insightful, is still correct and evergreen. There will always be money to be made in consulting… the question is who is going to make it and how.

Here is some of what I see looking around me and looking ahead, with some hat tips to others who are doing the same.

This has all happened before

I made the comment above back in 2001, when my consulting firm employer was between rounds of layoffs and many of our competitors were either closing their doors forever, merging with each other, or being unbundled from their previous corporate parents. Old-timers like me may still have t-shirts from that era with names like Viant, Scient, Monday, BearingPoint, General Magic, and marchFIRST.

It’s incredibly obvious to anyone in the space right now that we’re undergoing another period of rapid change in the consulting landscape. Writing in HBR in October 2013, Clayton Christensen, Dina Wang, and Derek van Bever nail the big takeaway:

“We have come to the conclusion that the same forces that disrupted so many businesses, from steel to publishing, are starting to reshape the world of consulting.” (“Consulting on the Edge of Disruption”)

The most visible forces are macroeconomic, the results of which mirror what’s going on across much of the private sector:

• Near the top of the consulting food chain, we’re seeing a lot of value consolidation through mergers and acquisitions. For example, this year two of my past employers—Stone Yamashita Partners and Sapient—have been acquired by larger firms. PwC and Booz&Company have merged into Strategy&. In recent memory, independent, marquee firms in the Bay Area like Hot Studio, One & Company, Pivotal Labs, and Adaptive Path have all been absorbed into enterprise clients (a kind of exit that a decade ago would have been highly unusual).

• At the tippy-top of the food chain, the “big fish” consulting firms are having a crisis of relevance, which is what you would expect from any massive organizations in times of rapid change. According to a recent New York Times piece, McKinsey is undergoing a particularly turbulent evolution as it responds to scandals and replaces long-held values with new, more stringent rules. Monitor Group has ceased to exist on its own. Some massive “full service” incumbents will weather the storm as always, but deep bench strength requires deep pockets, and accordingly the meat and potatoes clients of the big firms will always be Big Enterprise and Big Government, not the next generation of edgy startups. Success at the top of a food chain is somewhat like being the Red Queen: you have to run as fast as you can forever to stay where you already are.

• Meanwhile, as in other industries, new innovation in the consulting space is most obvious when looking at the “small fishes.” New and smaller firms are finding they can succeed by combining differentiated and sharply positioned services with innovative organizational design. Distributed teams and new org structures supported by the latest networked tools can sometimes deliver boutique results without the overhead of a traditional agency. My own firm The Next Us falls into this category, as do other organizations like Undercurrent, Neo, and Context Partners. Here’s prominent Seattle tech investor Chris Devore forecasting a broader trend of small agency innovation back in 2011:

“[I] see an opportunity for a “virtual industry” of boutique advisory firms, led by “partner-level” refugees from the old pyramids who understand how innovation happens now. These “innovation boutiques” will call in elite teams of specialists, assembled from the emerging class of high-performance talent marketplaces (think Forrst and Dribble for visual design, GroupTalent and oDesk for software execution, Contently for content creation, etc.), to deliver short-duration, high-impact solutions to their corporate clients.”

Other small firms are thriving by becoming exquisitely specialized. Think Droga5 for big brand ad creative, Outcast for media relations, and Kepler Group for marketing strategy, analytics and optimization.

So times are good for those who are correctly positioned. That said, while some consulting work is shifting or evolving, some of it is indeed disappearing by going in-house. As part of the overall value consolidation trend, Apple has decided they can handle advertising on its own thank you very much, and Netflix and Kellogg are two of many, many companies who have decided to take their programmatic buying internally.

Agencies will still be a vital part of the ecosystem, but their assignments may be narrower and less lucrative than before. Specialization and commoditization often go hand-in-hand.

And it will all happen again

Is there a chance in these current throes of change, or near-future ones, that consulting could effectively end entirely? Many might greet such an ending happily. In the novel Life, the Universe and Everything, Douglas Adams described an advanced alien civilization that decided to maroon its telephone sanitizers, management consultants and public relations professionals on another planet. The perception that consultants are by definition useless persists in some quarters to this day.

But—and I admit I can’t be objective here—I think there will always be consultants because the roots of the consulting industry aren’t in temporary macroeconomic or talent trends but instead basic human psychology. For at least several decades more, the average human brain is still much smarter than the smartest networked machine. We need human insight to spark and execute new ideas and maintain transformative change. At the same time, the human brain gets bogged down in very predictable ways. In particular, the larger an organization, the dumber and more insular it gets. Dmitri Orlov talks about this tendency memorably and rather colorfully in his article “Understanding Organizational Stupidity.”

With more positivity, Paul Pangaro looked at organizational stagnation back in 2002 and in the wonderful Little Grey Book used the insights of cybernetics to illustrate the continual need for leaders to look outside their organizations for fresh ideas and perspectives.

Consultants fill this “outsider” role perfectly. Writing more recently, Venkatesh Rao asserts that the primary value of a consultant is indeed his or her outsider status combined with the ability to safely comment on and sometimes attack the internal organization’s sacred beliefs:

“What the ideal consultant really brings to the party is a lack of a sense of the sacred. This implies a lack of a sense of the profane as well, and a lack of unexamined trust in the ideas and reasoning patterns of insiders. This state is not easy to achieve. You have to get practiced at systematically dismantling notions of sacredness, both the client’s and your own, to get to a sufficiently clean slate.

So the consultant is often just an outsider who is roughly as smart as the insiders who are hiring him or her, but processes differently by virtue of a missing or dismantled sacredness module.“

This is the same reason therapy persists as a profession. Our brains are wired to seek clarity from people who are outside our group and have the requisite training not to get caught up in our internal story.

The connection between business, personal, and social transformation is in fact a strong and enduring one. In his book The Age of Heretics: A History of the Radical Thinkers Who Reinvented Corporate Management (2nd edition), Art Kleiner of strategy+business places consultants in a multi-century lineage of heretics and outcasts who have led disruptive change. And he describes in detail how many specific trends in 20th century management consulting drew inspiration from social phenomena like the human potential movement, communes, Buddhism, and LSD. (For more quality writing on this theme, I recommend Steve Silberman’s retrospective of Steve Jobs, as well as Fred Turner’s From Counterculture to Cyberculture.)

Kleiner describes an almost-linear progression of 20th century heretical ideas— from balanced scorecard and scenario planning to quality management and lean manufacturing to re-engineering and knowledge management— each of which was absorbed, metabolized, bastardized, and discarded by the general business culture. Following this line into the 21st century, we could add things like “innovation,” lean startup, mindfulness, and resilience. These ideas are all worthy, but what’s notable of course is that they all have expiration dates. Once the intended clients have grokked the essential concept, and incorporated it as Paul Pangaro describes, there’s nothing for an idea-centered consulting firm left to sell, perhaps other than simple training.

So what’s new this time?

For that reason, I’m hesitant to say that the future of consulting is all about resilience, lean startup, Big Data, crowdsourcing, or any such theme that will have its moment in the sun before we move onto the next unsolved problem and genuine new idea. I will say that some new ideas are better than others. E.g., Venkat at Ribbonfarm has skewered “design thinking” and sharply critiqued lean startup—his essay on innovation methodology generally, “Product Driven Versus Customer Driven,” is still my favorite business read this year.

But the general list of trends I expect to bear most fruit include:

• Emphasis on “zero to one” value creation over optimization and incrementalism

• Increasing specialization and unbundling of services

• Deeper collaboration between and integration of skill sets: e.g., ad creative and buying, product and sales, marketing and IT, data science and everything

• Business models that can permit slower growth

• Increasing hybridization of services and products. (On this general theme, Aaron Dignan and Bud Caddell of Undercurrent note that for today’s organizations “Doing is cheaper than planning.”)

• GitHub-like idea exchanges, perhaps replacing 20th century conferences. (Twitter, Reddit, and Slack to an extent are filling this role already.)

• Overall, more activity in liminal spaces between the salaried world and the consulting world, with more porous activity at the membrane, including things like: highly involved angel investors; consultant/salaried hybrid roles; programs like Designer Fund’s Bridge that prepare freelancers for startup culture by immersing them deeply and thoughtfully in those environments; aggressive transparency à la Buffer, thereby blurring the distinction between what’s “inside” vs.“outside.”

After 17+ years in the consulting world and 41+ on Planet Earth, I’m old enough to remember where we’ve been and still be excited about the future. I’m looking forward to it.