Recalculate the Future of Law
Dan Katz, associate professor at Michigan State University College of Law and co-director of the ReInvent Law Laboratory, talked with Andrew Benedict-Nelson of Insight Labs about legal startups, incentivizing innovation, and what he tells lawyers who “don’t do math.”
Andrew Benedict-Nelson: I'd like to do a little Rip Van Winkle experiment with you. Let's say you fall asleep for ten years. It turns out you've fallen asleep in a law firm. What do you imagine the place is like when you wake up and look around? How would it be different?
Daniel Katz: What type of law firm? I don't think broad generalizations are usually right. I don't think the strategy for a member of the AmLaw 200 is the same as a mid-size regional firm or a single hangar. The market is pretty diverse.
Andrew Benedict-Nelson: Let's start with an easy limit on the exercise. You should pick a type of firm that will still exist ten years from now.
Daniel Katz: Well, prestigious firms like Cravath will still exist. The ratio of partners to associates may look different. But they've captured a certain segment of work, and it's hard to see how it would get yanked out of their hands. Sort of like Yale will probably be the last school to be hit by change.
Andrew Benedict-Nelson: Okay, so let's modify this a bit. Let's say you're not Rip Van Winkle but Washington Irving. What would be the most interesting kind of firm to catch a glimpse of ten years from now?
Daniel Katz: It would be the next level down, the niche-building firms. It would be the AmLaw 200 below the top 50. A great deal of legal work will be subject to automation — that's where that's going to happen, that's where the impact is going to be felt. They may be able to capture some of the top 50's work because automation will enable them to compete on price. So I think it would be interesting to look at a firm of that size, because it could go really well but also go really badly.
There's a generational factor too. A lot of those firms are run by people who by 2023 will not even be in the building. They will be on a beach somewhere – perhaps in Florida. That raises questions about for whom these organizations are actually being managed. Is it being managed for the 44-year-old who is about to have a first chance to be an equity partner? Or is it being managed for the 57-year-old who in ten years is going to be getting a gold watch? I would say that almost exclusively, it's being managed for the 57-year-old. The interesting question is what the 44-year-old is going to do about that. ...
No one is good at admitting that they don't know about something. It's easier to say, "That stuff doesn't matter. We're doing okay right now."
Andrew Benedict-Nelson: Let's do a little Turing test. Let's say you've got two firms, and one is the extreme of being managed for the sake of those older folks, and the other is the extreme of being managed for the upcoming generation. If you don't know anything about the ages of the folks who are there, what behaviors would you look at to differentiate between the two?
Daniel Katz: You'd look at the technology investment. You'd look at investment in process. You'd look at the fee structures. The idea that you're going to be measured by hourly rates — even if you're cutting those hourly rates — that's not an organization that's being managed for the future. Those are some of the hallmarks. I'm describing most firms, as you obviously know. If you really want to invest in efficiency of process, you'll tie yourself to the mast with a flat-fee arrangement where if you don't make the cut, you'll lose money. Those are the places that are going to figure out the innovations fast enough, because necessity is the mother of innovation. You've got to force yourself to innovate.
In those surveys of managing partners that they do, efficiency is usually about the 27th concern on their list. That's the problem.
Andrew Benedict-Nelson: I'm trying to think of other organizations that seems to have this persistent problem, this lack of awareness of how tomorrow looks not just for the personnel, but for the organization.
Daniel Katz: Partnership is the problem. Partnership is the acute difference in law. It's not that corporations don't have their problems, managing to the share price and all that — they do. But a partnership can literally vote itself more compensation, call it an investment, and say, "We don't care what the world looks like in 2023." No one is doing it that overtly, but that is the relative balance of things. The people making the decisions don't have the right incentives. Now imagine how quickly efficiency would move up their list of concerns if I said, "We just landed a $10 million fixed-fee arrangement, and if we can do it for seven, we get to keep three million bucks. But if we do it for thirteen, we eat three million bucks." Everybody's going to start caring about process and efficiency real fast.
I'm not trying to ascribe negative motivations to any individual in the current system. I'm just saying you think differently when it's your money on the line.
Andrew Benedict-Nelson: You know, for a completely different project that Insight Labs was working on, I interviewed a behavioral psychologist about creativity in corporations. I asked him what the typical CEO dreams of when he dreams of his ideal creative employee. He was very clear with me — he said the CEO doesn't have that dream. The CEO only dreams in terms of the organization's metrics.
I think that with law firms, you're seeing the lack of someone who is dreaming in that particular way. They're not dreaming as the embodiment of the organization. They think about things a little differently in a partnership structure, don't they?
Daniel Katz: Yeah, I'd say it's difficult for them to get outside themselves in the way you describe. It's hard to say to everyone, "We could be better than we are." But right now you're asking a person who's a millionaire to admit that there's something wrong. That is something that Richard Susskind has highlighted many times. Let's say you're a partner and you make $600,000 a year, and a bad year is $500,000 and a good year is $750,000. You're in the top one-half of one percent of wage-earners. It doesn't sound like you have a problem. Every signal from the outside says you're okay.
Andrew Benedict-Nelson: I also interviewed a theater designer not too long ago. He said "disorientation is a necessary element of design." You don't just need to be smart to re-design an organization — you have to feel some of that disorientation. You have to get to the place you just described: "How could we be better?" And not just a little bit better, but radically better. It sounds as if there's no place where that spirit lives at most firms.
Daniel Katz: That's right. You're basically asking, "Why don't law firms have R&D departments?" Some of them are getting those, but that's a very recent development. And the reason they're doing it is that they found some way to monetize it — they've figured out some innovation and they've found a way to sell that innovation, either through marketing or some sort of third-party arrangement. That's probably the most interesting development in the market right now for me.
If you're a partnership, you can't really raise capital, not in the United States. Lawyers can't share profits with non-lawyers. So if you have a good idea, you can't really monetize it in the way you would in other industries. Because you're stuck with your partners and your client list, your idea can never really achieve scale.
So this sort of thing is either coming out of firms that by some miracle have figured it out or who are startups themselves. They're the ones who are saying, yes, we have a partnership and the partnership will deliver the actual legal services, but we also have a company that is owned by the partners that can raise capital and sell technology. So that company isn't splitting the profits gained by legal services. … There are companies that are doing things like developing technological platforms first, raising money for that, and then only afterwards forming the partnership that provides the legal services.
Andrew Benedict-Nelson: I'm glad you brought this up, because we and the folks we're working with as a part of Law2023 have been talking about R&D for a while now. We're talking about it as a design criterion for our firm of the future.
Daniel Katz: Yeah, sure, of course. But recently I've been thinking that may not make any sense, because the issue has been left for so long. The startup community is growing so rapidly that they may be taking over the R&D function for the industry. You can imagine a world in which it was developed by firms, but it just wasn't.
Andrew Benedict-Nelson: That's just what I'm interested in, this structural question. For example, in the UK and other countries, reforms have occurred that make different ownership structures possible in the legal industry, and they're seeing firms with different structures as a result. So what do you think the structural story will be here in 2023? Do you think that we might also see reforms that would allow for firms with more developed innovation departments?
Daniel Katz: They could do it right now! There's no legal impediment stopping them. But they don't have the right attitude. They think, "I'm here to be a lawyer, not to do this other stuff." They often don't have an engineering bone in their bodies. When they have the same problem fifty times, they don't say, "How can we solve this? How can we invest in a process so we only have to solve it one time?" It's not how they operate.
We try to inculcate students with some of that perspective, with the idea that the repetitious use of time on something you could automate is silly. But it's not so silly when you make your money off of it, when you have to cannibalize it today to prepare for tomorrow.
Andrew Benedict-Nelson: I want to know more about the lawyer's brain. The observation you made about engineering, we've made about design — there's some sort of disincentive there.
Daniel Katz: Well, we're going for left brain-right brain lawyers. We have four things we're pushing: substantive law, technology, design, and delivery (which is business models, basically). Those are our four pillars.
Andrew Benedict-Nelson: And it's the combination of those things that matters. I imagine it was a conscious choice to emphasize that you need the whole sequence.
Daniel Katz: The whole ensemble, yes. What we have now are a lot of people who only have substantive legal expertise. That's not a precious resource by any stretch of the imagination, especially compared to the amount of work there is to do in the world. If we had 100 units of work available, there are a lot of people who can do a lot of those units. Some of them may be more specialized in some units than others.
But if you're tied for substantive legal knowledge, I'm going to go with whoever is better in those other elements, hands down, you win. If I don't really know which lawyer is better, I'm going to go with the one that has better technology or a well-designed process. I'm going to be looking someone with a business model that aligns my interests with theirs. In fact, general counsels are going so far as to trade off a lawyer that they may not think is the best for savings in one of these other areas. …
To say that our ability to assess the quality of a lawyer is noisy is the understatement of the century. We have very fuzzy notions of who's good and who's bad. There are obvious differences and less obvious ones. If you can't choose on that point, these other factors become your point of differentiation, and a lot of times they're much easier to detect.
Andrew Benedict-Nelson: And it can serve as a signal for other things. I know that in my small experience of hiring lawyers, even for petty family-related things, I've placed a lot of stock in things like the website design, only because it might be a signal of how much they care to effectively design other things.
Daniel Katz: You're making a purchase in a situation where you can't measure quality, so you're using other signals, and that's reasonable enough. There is a lot more going on there than people would like to admit.
Andrew Benedict-Nelson: But if you were to promote that to the level of a general counsel picking a lawyer for a company, it would be a pretty sophisticated decisions.
Daniel Katz: It would be, though a lot of them don't have any kind of analytical framework to do it. In many cases the operating rules is, "We've always sent this kind of work to Bob, we're comfortable with Bob," whatever that means.
Andrew Benedict-Nelson: It sounds like the way people bought car insurance or home insurance twenty years ago. "Let me introduce you to my agent…" and so forth.
Daniel Katz: Then what changed is that people realized how much less expensive a product of the same quality could be. The problem here is how you match on quality — some of these firms live and breathe on the idea that most of us can't tell the difference. You hear a lot of talk of rating agencies and that sort of thing, but the problem is that most of the metrics people use just aren't serious enough.
It always starts that way, though. You have to burn through the bad metrics to get to the good. We have to start by asking, "Why do I think this lawyer is good?" Having a good website doesn't necessarily tell you whether this person is going to help you in a family law case, but you may use it to make your decision.
That's why one of the most interesting parts of this whole thing will be retail. It's not what most lawyers are thinking about — most of them are thinking about the future of Big Law.
Andrew Benedict-Nelson: When I talked with Andy Daws from Riverview, we imagined a world where a lawyer is as accessible to people as the "mayday button" on the new Kindle Fire. We talked about the fact that that is not really a technological development — it's the end of a long series of incentives that Amazon has built into their own business environment. They're incentivized to build ridiculously good customer service because every minute you spend inside that product could make them more money through various kinds of downloads.
Daniel Katz: Keep buyin' those books, guys.
Andrew Benedict-Nelson: Yeah. So I'm actually quite interested in retail now, because I'd like to imagine what the similar "packaging" might be for ridiculously good legal services in the future.
Daniel Katz: It depends on what the consumer device would be. We had a conference in Silicon Valley in March where one of my students named Andy Ninh gave a presentation on on-demand services through Google Glass. A lot of this depends on your theory of what the dominant content delivery device will be. There are obviously a lot of people trying to make a play on mobile, but mobile itself won't stay in the form we now know.
What you'd need to think about are situations like a car accident — could someone press the equivalent of the OnStar button and talk with a lawyer about what they're supposed to do now? The strong version of this would be "wearable law" like "wearable computing." It's pretty far down the spectrum of artificial intelligence, but that would be a world where you have devices that are receiving and analyzing data all the time and that you call up when you need to.
Andrew Benedict-Nelson: What seems really important about that imagined experience is the cognitive component, of imagining the voice of the lawyer actually guiding you through experiences in life. I'm trying to think of an equivalent. I hate to say it, but the example that comes to mind is funeral parlors — they've figured out how to guide you through a very specific point in the grieving process really well, and have built an industry around it. They seem to understand not just you and your situation, but the general path that you're following and how it feels.
Daniel Katz: That comes from the fact that they've done it thousands of times. That's why having a lot of data would help so much in the law — you could train people to see the arc of experience through data, so they don't have to go through it thousands of times themselves in order to understand it.
Andrew Benedict-Nelson: Right, because a funeral home experience lasts a week, maybe two weeks, but a case can last years. Data is your only hope for developing a rigorous understanding of the experience in the abstract.
Daniel Katz: So that's more of a retail-facing issue, but I think it's part of the puzzle. I think it will be interesting to see how the lessons learned in one area of legal will apply to another.
Going back to the Rip Van Winkle game, I think one of the most interesting things to see over the next ten years will be the behavior of general counsels. What they decide will decide the whole ballgame in many cases. Once you've seen the case for a single automation or improvement in the process, there's no going back. You start to see the holes in a lot of other stuff. You feel more confident saying, "I'm not going to pay for that." You start thinking about how you can use what you've learned in your own organization to drive down costs. You think about how this might help you hold on to your job or get the promotion to general counsel, if you're a subordinate.
Andrew Benedict-Nelson: I'd like to ask you a somewhat more affective question. We've been talking about the need for lawyers to think like engineers, to think like designers. On a gut level, why do you think that they don't right now?
Daniel Katz: There's a lot of reasons for that. We like to ask if there is going to be math on the exam. There is the training and the socialization. There is a selection effect, a treatment effect, and an ongoing treatment effect that encourages a humanistic take on everything. It's this view that what we do is special, that it can't be understood by metrics — eventually we start to believe those things. Start with the selection — the type of people who could be medical students don't go to law school, as a general matter. Then there's the training, which feels more like Swarthmore than MIT all the way through.
I'm currently writing a paper that imagines the "MIT School of Law" as a thought-experiment. Because people are always talking about how they're going to reorganize law schools entirely around practical training. I think this is a bad idea and will not be successful. They need to realize that there's already a better model out there — it's called polytechnic. No one is going to accuse MIT of not being a world-class school. No one is going to accuse them of being insufficiently theoretical. They've just built a model where you spend your time asking, "What theories are relevant for what I'm trying to do in my profession?" Less Foucault and more Claude Shannon – that’s for damn sure.
Andrew Benedict-Nelson: This squares with my own experience. I was a strong humanities student — that was my thing. Then when I was 22 or 23, all these people started asking me if I was going to go to law school.
Daniel Katz: I'm not against the humanities at all, especially for undergrads — I just want to make that clear. But we're talking about professional education.
Andrew Benedict-Nelson: Right, this is what I'm getting at. I think a lot of people who had undergraduate experiences like mine are thinking, "What the heck does going to law school have to do with the fact that I earned Honors in History?" But then they go! I guess what interests me about it is that you're talking about selection and socialization rather than any kind of theory of how all that material relates to the practice of law.
Daniel Katz: They're taking some things you were good at and connecting them to a high-status profession, or at least what people think of as a high-status profession. It sounds very flattering. It says, "Oh, you'd be good at that."
Andrew Benedict-Nelson: But no one ever says to the kid who wins the undergraduate English paper contest, "Oh, I know, you should be a physician."
Daniel Katz: So that's how we get the population, and then we do nothing to train them in a way that counteracts that. That's the treatment effect. And then the profession itself is dominated by that perspective as well. Breaking with that perspective is perceived to be illegitimate or unsound.
Andrew Benedict-Nelson: It has the attributes of a paradigm. It's not just that they're looking for English majors… they think of what they do and what they know as being "coded" in a certain system, and it's not a quantitative one. And it comes across in their style of thinking.
Daniel Katz: This is how nuts it is. I teach a course in criminal procedure. Let's say I raise a problem that involves a calculation with a fraction, say one-third versus one-half of something. I have students who say, "Well, I'm not good at math." Now, what exactly are you saying? Are you saying you don't know what a fraction is? That's probably not the issue. The issue is that you've been operating a world in which it was fine to just assert that, to say, "We don't do math here." You would never say, "Sorry, I don't know anything about the case we just read." You would figure it out.
Because in the end, do you want to say a client, "I don't do math"? Is that how you want to represent yourself? Right now we're telling a lot of students that's okay.
Andrew Benedict-Nelson: I think it helps in that scenario to think about math as a technology. Here's what I mean. My wife is a whiz at shopping for antiques. She can walk down the aisle in an antique store and rapidly discern from hundreds of items whether there's a good deal or not. Now, by her own admission, she's not a math genius. But she's sort of built a machine in her head where she's integrated price into her perception of reality.
I feel like that's what's missing from lawyers in your account. It's not that we've built the legal profession from a population of people who flunked algebra — they could do math if they tried. It's that they don't habitually use it as a technology.
Daniel Katz: Right. But culturally it's a completely accepted thing. I've had faculty members say that sort of thing to me. Not from my own school — they know I would give them a hard time.. But I'm tired of people saying things like, "I didn't go to law school to do math." They think they can just put this footnote or asterisk ahead of quantitative reasoning and that's sufficient. The world is going to show them that that is an incorrect statement.
In fact, I'm quite thrilled to have a field that thinks that's okay, because that's where all the opportunities lie for us. I want other schools to not doing anything about this, to keep on keeping on. There has been a study going around on how a JD has a $1 million return. I want people at other schools to read that, believe that, commit it to memory, and not do anything. They love that study so much because they see it as a justification for the status quo.
Andrew Benedict-Nelson: If that turns out to be wrong, it will be a very expensive mistake for the last person to believe it's right.
Daniel Katz: Right. But let's say it's not wrong (which it is). Let's say you keep getting a decent return. That's not to say that someone else can't provide an even better return by innovating way past you. That's not to say that the entire industry isn't exposed to competition from someone who could do it much better.
Andrew Benedict-Nelson: I think this gets at another important aspect of the culture of the profession. There's a chapter in The Black Swan where the author is considering whether he would tell a young person to follow a career path through "Mediocristan" — professional careers like law or medicine where the highest-paid folks don't make much more than average — or "Extremistan," which is fields like finance or technology where the highest-paid folks make an exponentially greater amount. He says that statistically, you have to recommend "Mediocristan."
So lawyers have made this choice. They think of themselves as accomplished individuals who ought to be well-compensated, but they don't think of themselves as making plays to win the entire game, to dominate the entire field. What you're saying is that the law could now include some people who are motivated to undertake those kinds of risks as well as a population of folks who are happy to just keep making six figures.
Daniel Katz: That's true. And that kind of imbalance has led to a kind of retrenchment in many law schools. But I couldn't be happier about it. I'd be happy for the system to keep generating studies that encourage folks who already have market share to stand down.
Andrew Benedict-Nelson: Let's talk about that. If you're in the future and looking back at your own program, what are some of the things you hope you'll be proud of or excited about?
Daniel Katz: I'm happy to see people starting to mimic some of the things we've been doing. But a lot of people outside the "insurgent" population don't know what to make of us. We think we've planted some really good seeds, though, among companies that will actually grow in the future. We've made inroads where we might not have been able to otherwise.
Andrew Benedict-Nelson: One more question based on the amount of time you spend in the legal innovation space. What's something you're sick of hearing about? What's an overrated aspect of this conversation that you hope our group just avoids altogether?
Daniel Katz: There are a lot of things I'm sick of hearing about, but it doesn't mean they're wrong. It's more than I'm tired of going over the same ideas fifty times with people. I've given the same talk on the science of prediction in law so many times and yet I reach people who go through the typical arc of telling me it is impossible to telling me it was actually their idea in the first place.
Andrew Benedict-Nelson: What I take from that is that you'd like to be hearing more verbs and fewer nouns.
Daniel Katz: That's right.
Andrew Benedict-Nelson: From where you sit — certainly from where I sit — hearing a story of how a firm struggled with a concept like prediction and screwed it up and learned something would be so much more interesting than going to another conference with a keynote on prediction.
Daniel Katz: Yeah. But in 2014, we're going to be teaching a class on legal analytics where students are actually going to be asked to predict things using machine learning. Because I'm less interested in talking and more interesting in doing stuff.