Oh, the mistakes you will make!
After months of getting nowhere, I decided to shut Sentinel down. In this post, I reflect on the mistakes that I made, the learnings, the challenges, and how you could make it work.
It’s been a while since I last wrote something here. In that last post, I wrote about a new project I was embarking on called Sentinel Healthcare. Well, fast forward to today and unfortunately, I’ve since decided to shut down the current approach and re-evaluate things.
It obviously sucks to fail. No one sets out to come up short of their goals. However, failure presents an opportunity. An opportunity to reflect on what went wrong and learn from it. And yes, I’ll admit, they can also be a bit therapeutic. That’s what I’ve attempted to do here.
Since it ended up being a lot of content, I broke it down into different sections. Feel free to consume this however you want:
Here it goes…
Some background on Sentinel
I won't go into all the details behind why I started Sentinel. I touch on that more in my initial post about it here. But, I do think it's worth providing some of that context for this reflection.
I started Sentinel to solve some of my own problems that I was experiencing with the current healthcare system. Those problems mostly centered around two main pain points: 1) the current primary care system leaves a lot to be desired in the modern age, especially for the young and relatively healthy and 2) healthcare suffers from a significant asymmetry of information, maybe more than any other service we interact with. I also felt like it was the right time to pursue solving these problems because of the capabilities I was seeing become available through generative AI. Specifically, the ability to use it to create a modern and easy to use product, the ability to democratize access to otherwise hard to get information, and the ability to help the average person make sense of a massive amount of personal health data that their devices were collecting.
This led me to building an iPhone application that allowed users to connect all of their health data (activity data plus health records) in one place, give them a personalized AI health assistant that had access to your data in order to answer health questions specifically for you, and allow them to purchase their own labs so that you could gather further personalized health data that wasn’t being collected by your devices and where you did not have to go through a doctor to do so.
In the end, the product never gained much traction and was pretty expensive to keep running. Since I was bootstrapping it from my own savings, I decided to cut my losses and shut it down while I considered other approaches and made a decision on whether or not to move on to other things.
In the rest of this post, I’ll detail some of the biggest mistakes that I made and how I could’ve potentially avoided them. My hope is that by sharing all of this, someone somewhere may learn a thing or two that gives them a higher probability of success at whatever it is they’re starting. Hopefully my experience, mistakes, and learnings set someone else on the right path. If that happens, then all of it was worth it.
Mistakes that I made
Oh the mistakes you will make! Maybe that should’ve been the title of the Dr. Seuss book that seemingly everyone gets as a gift at their graduation. It would at least be an accurate reflection of my experience with starting companies. I sure made a whole host of them with Sentinel.
But if I’m 100% honest with you, it’s impossible to not make mistakes. Mistakes are also healthy. They mean you are taking risks, trying to do new things, and (hopefully) learning from them. Even though it seems everyone learns more by making their own mistakes, hopefully by sharing these someone else might be able to avoid them.
Mistake #1: Going from technology to problem
It’s almost always a pitfall to go from technology to problem. This mistake is extra painful to me because I was already aware of this pitfall, yet I still suffered from it. That’s how difficult it can be for someone like myself, who is technical and enjoys playing with new tools, to avoid this issue. In Sentinel’s case, the mistake that I made was taking the technology (generative AI) and go hunting for a problem that it could solve.
Now, I think it’s important to caveat this mistake. It is very possible to make this work, to go from a technology to a problem. In fact, I’d argue ChatGPT is exactly this type. They pursued artificial general intelligence and through that pursuit came upon something that could solve people’s problems1. And there are other examples of this throughout the technology industry (many of Elon Musk’s companies, for example). So, I’m not saying that it can’t work, but rather it was a mistake for me to do it in this particular instance.
The reason I feel like this was a big mistake by me is not that it couldn’t have been pulled off, but rather, in order to pull it off, it would take a lot of time and money to do. If you notice, a lot of the people or companies that pull off going from a technology to a problem, already are successful and have the reputation and funds to take these moonshot attempts. That isn’t me. And this approach is especially hard in an industry like healthcare, as healthcare is more technology averse than technology forward (more on that below).
It’s very possible that I would’ve decided that generative AI was the right approach to solve this problem in the end, but by starting with the technology first and finding a problem in which it could fit, I actually ended up causing myself to face a number of additional challenges that I might not otherwise have had to face had I started with the problem instead. For example (and I’ll get into more details of these below), I might not ever have had an issue with getting the app approved by Apple if I had started with the problem. By starting with the technology, I knew that these models would do best with lots of personalized data, so I looked for things I could leverage to get that data. This led me to the discovery of Apple’s HealthKit Health Records SDK, which led me down the path to building an app that had this at its core.
Leading with the technology rather than the problem, also led me to skip steps along the way to defining the product for the target market because I already had it in my mind that this particular solution would be best. Maybe it was, but by skipping this discovery aspect, I may have missed some better approaches that could have gotten me to test my core hypothesis faster.
Mistake #2: Pushing vs pulling towards an idea
This mistake is related to the first mistake. It might also be a bit more controversial, so let me explain what I mean by it and why I think it was a mistake in my case.
When I say I pushed myself towards this idea, rather than being pulled towards it, what I mean is that I was the one pushing the idea, I was not being pulled into the idea by either the market or someone else. For example, it was my own pain point that I was trying to solve. While I believed this to be a shared pain point, there was never a time along the way that anyone else was begging for me to do this. I was the one pushing the product and concept onto others, rather than them pulling me towards a particular solution. You’re always going to need to push when you start something new, that’s a given. But it should be a red flag to you if along the way nobody is pulling what you’re trying to push.
The reason it could be controversial to call this a mistake is that the world needs more entrepreneurs. If they didn’t exist, everything would remain stagnant and we’d have no new ideas or things to look forward to. The economy and life would be pretty meh. On the flip side of this, building a company from scratch is just a freaking hard thing to do. So, the world needs people to believe they can overcome the odds and start things that have a high likelihood of failure. We need to push more people into it2.
In fact, I would guess most people probably fall on the side of being afraid to try rather than maybe overdoing it. My advice here might be that you should probably do the thing that feels hard for you to do. So, if you’re afraid to make the leap and have never done it, you should probably do it. You’re likely holding back too much. But, if you’ve tried to do it a bunch and haven’t quite succeeded, maybe it’s time to reconsider the approach and wait for more pull or signal before leaping.
Bringing this back to my own story, I can tell you that I’ve tried starting around 4 businesses so far and really only had two successful outcomes. One was a business that basically fell into my lap, where I got pulled into it because someone asked me to do something for them, which led to another and another and became a successful business I ran for multiple years until I decided to do something else. The other didn’t even start as a business. I built a small tool for myself to import some stock data into Google Sheets and then turned that into a Google Sheet add-on for others that I asked people to give me $5/month for. Pretty much since the first day that tool launched, people were paying me for it and asking for more features to be added. I ended up selling it to another owner in the end.
All the other ideas have been flops that I started as a way to build a business because I thought it would be a good idea. It’s 100% true that all of them required effort on my part and a lot of pushing at times but the ones that were most successful felt much more like I was being pulled then I was pushing3.
The last thing I’ll say on this is this is likely more of a mistake for bootstrapped businesses than investor backed ones. The reason for this is because a bootstrapped business typically cannot survive very long when it’s self-funded4. This means that your timelines and pressure for finding product/market fit are heightened. Timelines and pressure can be a positive thing, don’t misinterpret me here, but they don’t make it any easier on you to find something that works. Bootstrapped businesses are better started as side projects that pull you in later by how successful they become. Venture backed businesses can have more of the “we’ll figure it out as we go” element because you have more capital to burn to do it and you’re not pressured to make money right away5.
In the end, I don’t want to make this seem like I’m discouraging others from taking the leap into starting a business that you think is worth it, you definitely should do it! If you’ve got a burning passion to see something exist in the world, make it happen. But, if you just think you’ve got a good idea for a business that’s it, just take some caution here as it’s going to be a struggle6. That’s not to say you cannot do it and all the more power to you if you can, but starting a business is already a huge challenge, having more things going your way at the start can help a lot.
Mistake #3: Assuming I knew the market well because I was in it
This mistake might be the most egregious mistake that I made as a former product manager. Intuition and instincts only get you so far. I made the assumption that because I was building a product for myself, I knew what other people like me wanted. While this wasn’t entirely inaccurate (many of the pain points I had were common amongst others like me), I should’ve done more homework before diving into it and not making assumptions. This could’ve led me down a different path or to realize I was on the wrong path earlier than I did.
Did I fully understand the self-insured target market? I thought I did pretty well because I experienced it first hand. But this wasn’t enough, I should’ve known it in and out, including the challenges that I might face along the way around health and distribution. There’s not a whole lot more to say on this other than spending more time researching and doing things that seem small7 rather than just diving into building things because I was the customer.
Mistake #4: Jumping to the build stage too early
This mistake relates to mistakes 1 and 3. Because I started with the technology as a primary driver and I made assumptions about the market, I jumped to the build stage too early. Specifically, what I mean by build stage is building the actual product. This is an important distinction because no matter what your idea is, you’re going to have to build something. That might be a landing page, a survey, etc. Whatever it is, you’re going to have to build something. However, I think it was a mistake for me to start to build out the product as early as I did.
If I were to justify and defend my decision for why I did it the way I did, my reasoning would be (outside of what I already went through above) that at the end of the day, a startup is all about testing hypotheses. And in order to test your hypothesis that people will pay you for a product, you’re going to have to build that product.
I can already hear the chorus of lean startup people yelling at me that that’s what an MVP is for! Sure, it’s true that this is the idea behind building an MVP and design Sprints, etc. I’ve been around startups for awhile now, I’m aware of the different approaches and terminology. However, there’s a couple issues here.
First, "MVP" is sort of like a Rorschach test for the startup world, "when you hear the word 'MVP', what does it mean to you?". To me, the key part of figuring out what your MVP should be is in figuring out what the minimal way to test your core hypothesis is. That doesn’t mean "cheap" or "bad" or "small". It just means minimal. The second issue is that everything is context dependent. Minimal in one context is different from minimal in another.
My approach to the MVP in the context of Sentinel was to build out a simple version of the product that tested the core hypothesis: that people would be willing to pay for a self-directed consumer health product with an AI at its core. I believe that the mistake I made here was that I needed to build out a product to truly test this hypothesis. It’s sort of hard to evaluate whether or not this was a mistake in hindsight, since I could’ve used other approaches, like a landing page or ads or demo mockups or a combination of all of the above, but even if these didn’t get a lot of traction, would that have really said there wasn’t a market? Hard to tell in my opinion.
In fairness to me, I didn’t only build a product. I also built a landing page with a waitlist sign up and ran ads against it. All as ways to try and better test the core hypothesis. But I ran into a few problems with this approach. The first problem was (and still is, lol) I am a nobody. I’m not trying to beat myself up here but in the grand scheme of things, I really am a nobody. I have maybe a couple hundred followers across the different social networks, of which I can only presume about 5% of them actually consume any of the content I put out (which may be generous given social media consumption behavior). All of this means that I don’t have an easy distribution mechanism to get my ideas out in front of people. I’m no Mr. Beast, who could tweet a link out and get millions of people to click it and take action. Which means, I didn’t have a natural distribution channel to test my hypothesis, especially for a consumer product8.
To try and address this, I tried running ads to drive people to the website and get them to sign up for the waitlist. That worked fine but I was bootstrapping and felt like it wasn’t an efficient use of the money9. At the same time, as I was going deeper on the product side of things, I thought I could be clever and strategic by not only being able to utilize Apple’s HealthKit SDK to get the data that I needed on the product side, but also leverage Apple’s iOS App Store for distribution (which I’ve done successfully in the past)10. This made me believe that building the product could give me multiple advantages, it would let me test both the core hypothesis and a distribution channel.
I’ll get into more of the details around the whole iPhone app thing in a bit, but the short version of it was that this calculation ended up being wrong, resulting in a lot of time and money being wasted. The calculation to make the bet wasn’t the mistake in my opinion, but rather the mistake was in building this early on without more signal that I was onto something. By building too early, I missed other potential paths that I could’ve taken that would have avoided the issue entirely. Or, I could’ve had more confidence that it was the best approach to take.
I could probably go on and on about my reasoning here and defend the decision to build at least decently well, but in the end I do think it was a mistake and I was too biased towards product, being a product person. I could’ve done a bit more work on the distribution and marketing side to try and get more signal before deciding to build. Would that have actually changed my mind or led to another idea or direction? I’m not sure, it’s impossible to play the alternative realities out. But, I do think I could’ve done more pre-product than I did, and that was a mistake.
Things that I learned
Although I made a number of mistakes (the biggest of which I just described), I learned a whole lot too. Obviously, I learned from the mistakes that I made but through research, development, and putting something out there, I learned a number of additional things that I thought would be worth sharing here too. Specifically, I learned a whole lot about the healthcare system and the ecosystem that has been built up around consumer healthcare.
The Healthcare System
Let’s start with what I learned about the healthcare system. To set some context, I do not have much of a healthcare background. A number of years ago, while running my own software consultancy business, I helped a few healthcare startups with their products, so I had some experience with it. But this experience was mostly one level removed (I mostly worked on the development side, not on the business and customer side) and specific to the internal workings of hospitals, which is much different than consumer health.
The only other connection that I have to healthcare is that I married into a medical family. Both of my in-laws are doctors and a lot of the initial thinking around Sentinel came from my experience with them. In particular, I always felt that it was a bit of a cheat code to have my in-laws for different medical situations that arose. More than once, they provided our family with insights, access, and caught issues that would’ve otherwise gone undetected or at least not detected initially. Without them, some of the medical outcomes for my family could have been very different, sometimes with really drastic results. This fueled some of the inspiration behind Sentinel. I wanted to provide the things my in-laws were providing me to everyone.
But beyond a brief experience with healthcare startups and my in-laws, I really didn’t know much about the healthcare system, especially the business dynamics of it. Of course, I read some things about it and had some general intuition but you never really understand a thing until you experience it first hand.
One of the first things that I learned was how not a big deal HIPAA compliance actually is. HIPAA compliance gets thrown around as either great policy by politicians, lawyers, and incumbents in healthcare or as something really bad and scary by startups and investors. The truth, like almost everything, lies probably somewhere in between. In my experience, HIPAA compliance, technically speaking, was not all that bad to follow or hard to implement. Long story short, HIPAA compliance for software companies can be boiled down to using security best practices and a lot of paperwork. That’s the positive side of it. The negative side of it comes down to costs. And the costs you incur from it aren’t always direct.
Probably my biggest overall learning about the healthcare system is this: everything in healthcare costs 10x more than the equivalent in almost any other industry. HIPAA compliance is partly to blame for this, but not all of it. The reason everything in healthcare costs 10x more than it does elsewhere is due to some interesting dynamics. First, as mentioned, is HIPAA compliance. But HIPAA compliance increases the cost, not because it is so difficult to implement, as I mentioned above, but because the particular requirements it has are not implemented by all software providers.
For example, one of the requirements of HIPAA (section 164.308(a)(4) to be exact) is that an organization must implement technical policies and procedures for accessing a system that holds electronic personal health information (ePHI). That sounds scary but in practice that just means having role based access controls built into your software. For large enterprise businesses, this is typically implemented for most of their products and services by default. However, it’s not always the case for every one of their products and services, especially new ones or ones that mostly serve startups.
In addition, startups who provide services to software companies, like cloud provider type services (database/data/server/etc hosts), also don’t always have this baked into their products. And why would they? It’s faster and easier to not have to do this right away. And once you don’t do it to start, it drops in priority to implement until it’s really needed by your customers. And the types of customers that really need these features are typically large Enterprise businesses. More on this in a second.
What this dynamic ends up doing is preventing healthcare companies from using any services or startups that haven’t implemented these policies. That means a healthcare company is by default choosing from a more limited number of providers for its various needs. And as Econ 101 tells us, if you reduce supply and keep demand constant, price will increase.
Another component for why everything is 10x more expensive in healthcare is related to the above, and that’s everything in healthcare is Enterprise oriented. What I mean by this is that because of the particular requirements (for which I mentioned one above but there are a number of others), it really only makes sense for companies to implement these features for Enterprise size businesses. Not only are those the only businesses that typically have those features as requirements, but it’s also more time consuming to build them out, which means higher costs and therefore higher prices too. And who can you charge higher prices? Enterprise businesses.
So, the thing you learn in trying to build healthcare software is that when you look at a vendor’s pricing page and you see their pricing tiers, you are almost always put into the Enterprise tier (aka Custom pricing, aka Sign up to talk to sales!). It’s much easier for companies to bucket healthcare companies into the same tier as Enterprise when they have many of the same feature requirements. It’d be too complicated to try and split these features and prices out into their own tier, just to make it cheaper for the smaller market of healthcare startups.
The last component for why everything is 10x more expensive in healthcare combines everything, and that’s that these higher prices make it difficult to start a new company with low costs. As a result of these high costs, you see a lot of consolidation happening in healthcare, which only exacerbates the issue by creating a few large companies that dominate everything.
As you may have gathered from the above, when everything costs 10x more and every vendor you have to deal with to build out your product charges you enterprise prices, it prevents a lot of new companies from being started. This is especially true if you wanted to bootstrap your business and/or keep your prices low, since you’d have to also keep your costs low.
This means a lot less small companies get started in healthcare, which means, you guessed it, more of a reason for vendors to only charge enterprise prices11. And if the costs are high and it’s a struggle to survive, what do you think a business ends up doing? They either die or they sell out to a bigger business. Further consolidation, which continues the cycle of high costs and more consolidation. And on and on.
It likely doesn’t require much imagination then to imagine how all of this causes higher prices being passed on to consumers. And I’m not even touching the terrible incentives that exist where the more you treat someone, the more money you make, which only encourages doctors, hospitals, and healthcare companies to over treat, increasing costs for a consumer even more.
The last thing I’ll mention that I learned about the healthcare system is that almost everyone in it has a disdain for health insurance. Obviously, health insurance is not popular amongst consumers, but I hadn’t really processed or thought it’d also be so unpopular amongst providers and vendors. Pretty much everyone in health hates insurance, except for insurance companies. There are likely a number of different reasons for that but from my experience, it mostly boils down to the added complexity of dealing with them12.
One of the odd results of this is that everyone in the healthcare business loves cash-based business models. That is, everyone except the consumer, who has to be the one to foot the bill in the end13. This just adds one more reason it’s so challenging to start a consumer healthcare company. It can definitely be done, but there are a number of things stacked against you that you need to be aware of.
The Consumer Healthcare B2B Cottage Industry
Another learning that I think is worth sharing is my discovery of what I’ll call the "Consumer Healthcare B2B Cottage Industry". It surprised me to discover that there’s been a small ecosystem that’s been created around serving a new generation of consumer healthcare businesses (companies like Ro, His/Hers, CostPlusDrugs.com, etc). Perhaps it shouldn’t have surprised me very much to discover that there were a number of companies that have been created to serve these businesses, since they’ve generated a lot of interest and revenue, enough that at least a few of them are now public companies.
I won’t list out each of the individual companies that I came across here (if you’re interested, drop a comment), but I will tell you what the different buckets these businesses fall into. This hopefully will give you a sense of how you could build a company on top of them.
The first bucket of startups in this cottage industry are what I’d call "doctors as a service". The basic idea behind these companies is that they rent out doctors to you depending on your business’s needs. For example, many of the most popular consumer healthcare companies today do things like sell drugs for particular conditions. In order to sell these drugs to a patient directly, you need a doctor to write the prescription. That’s where the "doctors as a service" comes in. They can take the personal information shared via an app or website and decide whether or not to write the patient a prescription. These services can also do a lot more beyond prescription writing, like ordering lab tests, consultations, etc. Most of them, however, are focused on prescription writing or Telehealth consultations because that’s what the consumer healthcare companies that are most popular today need.
The next bucket of startups fall into the "pharmacy as a service" category. These companies do exactly what you’d expect, they make it possible to manage and fulfill prescriptions and send drugs to patients’ homes. This type of business is what makes Mark Cuban’s CostPlusDrugs.com possible. In combination with the "doctors as a service" you can string together a platform that can write prescriptions for patients and send the drugs for that prescription to the patients home, all in custom branded packaging.
Finally, there is also a bucket of companies that offer "lab tests as a service". These companies typically both sit on top of the "doctors as a service" providers and string together a few other services, like lab provider backends, etc. These businesses enable the consumer healthcare businesses that you can order labs through and see the results. Adding this functionality has become a popular feature to add to many of the consumer healthcare startups today.
All of this is actually pretty cool in my opinion. It allows for healthcare startups to get off the ground and running much faster than would otherwise be possible, and with much lower risk. When I originally was thinking up Sentinel, I imagined that I’d have to hire a doctor to actually provide any sort of medical service. Instead, you can skip that step and avoid some of the risk with being a direct care provider yourself and use these businesses instead.
However, it’s not all just rainbows and butterflies, there are a few additional things to be aware of about these service providers. First, for all the reasons outlined previously and the fact that they are actually providing care and therefore bare a lot of the risk, these services are not cheap. The cheapest providers you will find start around $1k-2k per month and go up from there. Cheaper than hiring your own team of doctors, but not that cheap when you’re bootstrapping a business.
The other thing to be aware of is that they’re mostly oriented towards working with a particular type of business. I briefly touched on this above but because of the success and frankly demand for consumer healthcare products like weight loss drugs, sexual health drugs, and even COVID drugs, the large majority of these “behind the scenes” health companies have been honed to focus on prescription writing. Even where they will do more, like patient consultations, they often will work better if these are targeted consultations (for example, a specific condition) rather than open ended consultations, like you might have with your primary care provider or get at an urgent care center14.
This makes sense for their business model but can be a little more limiting in what you can offer your customers if you’re on a tighter budget. For example, when initially planning Sentinel’s service, I wanted to make it capable of providing everything that a primary care office would provide patients: consultations15, prescription management, lab tests, and doctor referrals. But after talking with a number of these providers I quickly learned that either I’d have costs far higher than I could afford, or they were just not interested in providing that generalized of a service. This made me narrow the focus of the initial service to just labs with an AI assistant, which likely also handicapped the product a bit16.
The other learning that I took away from all of this, which might be helpful to others, is that consumer healthcare companies are better thought of as marketing companies than as technology companies. Yes, these companies might use software to string together these services and have a website and an app, but at the end of the day, they are selling a brand. With the cottage industry that exists around providing many of the technical elements to running a healthcare business, the only way to really differentiate yourself from your competitors as a consumer healthcare company is through your brand. It’s not the technical capabilities that set you apart, but rather your branding and marketing.
Which brings me to the last learning that I’ll share about this cottage industry that I found surprising: almost no one in this industry today is thinking about or using AI. I’m sure they are being sold AI tools behind the scenes to run their day to day processes (there are a ton of B2B healthcare AI startups now), but when I talked to a number of them about what I was trying to do, they were both excited about the possibilities of using AI on top of their services and found it to be unique. The reason that this might have been the case I’ll touch on in the next section.
Challenges with AI in health
Trying to create a consumer health product that heavily relied on AI had a lot of different obstacles. Some were more surprising than others. The first major obstacle that you face when trying to bring an AI consumer healthcare product to market is that healthcare doesn’t really do "new" well. What I mean by this is that because healthcare is so personal and private, it comes with a lot of legal baggage. Of course it makes sense that there are laws to protect consumers for how a business or provider uses their personal health data. But the result of all of these laws, plus how easy it is to sue someone over them, is that the industry is very conservative.
That doesn’t exactly bode well for anyone trying something new, especially something at the cutting edge that few people truly understand like generative AI. It’s not as if lawyers are well known for moving fast on things. Startup founders and lawyers are almost the polar opposite of each other. This obstacle surfaced in a few different ways, but the major way in which I faced it with Sentinel was in trying to get the app approved on the iOS App Store.
The iOS review process
Let me start off this section by saying I questioned whether or not I should even include it in this post. The reason I wasn’t sure whether or not to include it is because I don’t want it to sound like I’m complaining about it. I’m including it because it’s true that a big reason Sentinel ultimately wasn’t going to make sense as I built it was because of the issues I had with getting the app approved. Had it been accepted by Apple, the product likely would have still struggled and may have ultimately been shut down anyway, but without the approval, I couldn’t know for sure.
I won’t go into a ton of details around the entire iOS App Store review process that I experienced with Sentinel, but in the end, it took over two months of being in review and going back and forth with Apple before Sentinel was ultimately rejected from the App Store.
In my opinion, the reasons boiled down to Apple’s misunderstanding of what our app did and how it worked. Unfortunately, though I tried to make our case as best as I could, there’s not much you can do if they disagree or see things differently. It’s their platform, their rules.
The reason for the rejection was said to be the violation of their Physical Harm policy. This was in relation to how Sentinel could diagnose based on symptoms given. They claimed that it didn’t reference the exact sources of where that information was coming from and therefore was in violation of these policies. They also claimed that because of this we needed to prove that the app had regulatory clearance to be used in the regions we were planning on distributing it.
It wasn’t clear to me how Sentinel was in violation of this policy. The specific callout on the assistant potentially giving medical information in the chat based on provided symptoms is little different to the exact functionality ChatGPT or other AI chatbots offer. If anything, Sentinel was more conservative in this regard then those others. The product had multiple disclaimers in it and required users to explicitly accept our Terms of Service, which also had an explicit disclaimer that it was not to be used as medical advice but for informational purposes only. In addition to that, any time you would try to ask it something related to medical advice, it would say in its message to you that you should speak to a medical professional.
When I asked Apple about this I never got a good answer. Instead, it just led to more review time and the same responses and back and forth with them, over and over.
I get it, Apple is trying to protect their users. They are the protectors of the iOS platform and if you want to distribute your product through their App Store to their users, you have to go through them and follow their rules. Even if I disagree with their assessment, it is what it is. Perhaps if I had a team of lawyers, I could’ve figured out what it is that they really wanted or better clarify how Sentinel works and gotten past the review process. But since that wasn’t going to happen and since getting regulatory clearance wasn’t either (I still don’t understand what this is all about), I had little choice but to decide this wasn’t the right approach.
The only difference I could think of between ChatGPT being approved in the App Store, yet not Sentinel (besides having their own team of lawyers), is that Sentinel was a healthcare app. By categorizing the app as a health and medical app and using Apple’s HealthKit Health Records SDK, Sentinel was being put under a different lens than other AI apps, even if the underlying technology and functionality that was being called into question was the same.
Apple just didn’t want to take the risk. I get their cost/benefit analysis. But this is one of the challenges with trying to do something new in healthcare, no one wants to take risks.
Ultimately, the failure to get it approved falls on my shoulders one way or another. I also was the one who decided to take the risk in so heavily leveraging iOS for the MVP of Sentinel. I took a risk, it didn’t work out. C’est la vie.
Hallucinations
Another challenge that I faced that was on the more surprising end of things trying to use AI in healthcare was the hallucination problem. The obvious hallucination problem that you’re likely thinking about is where you ask the assistant for something and it says something completely wrong. Well, the good news is that I never ran into this, at least that I was aware of. In all cases, through the beta testing and otherwise, the assistant was very good at taking information about a person and their questions and giving good answers to those questions. I tested and verified these responses in a whole host of different scenarios.
The thing, however, that it did hallucinate with that was extremely frustrating was that it would "forget" what it was capable of. For example, one of the features it had was the ability to ask it to review your lab tests. This could be your historical lab tests from your health records or it could be from the lab tests that you order through Sentinel. The basic way it worked was that the AI assistant was given access to a search function that would be able to run a semantic search on a user’s lab tests. This search could be filtered by category and dates.
When this feature was first introduced, it worked really well. I could send the assistant a message, like "can you tell me what my most recent lab tests were?" and it would query the lab tests, sorting by date and return a good response. There was a bit of time where it wouldn’t use the current date to search for results and instead use its knowledge cut-off date, but with enough tweaking of the prompt, I was able to overcome this issue consistently. As time went on, however, it started to get worse and worse at this.
To be clear, I changed nothing about the implementation, the search function, or its prompt. It just started not "understanding" that it could search a user’s lab tests. So, if you asked, "can you tell me what my TSH levels were in my most recent results?" it would say something like "sorry but I don’t have access to your test results, please consult your doctor". This style of hallucination, where it stops working after it was working, is extremely difficult to deal with. It’s one thing if it stops working after changes are made to the model you use, to the prompt you use, or to the search function. All of that is both understandable and fixable, since you can either go back to the way it was or update things to compensate. But when you give the API the exact same thing and nothing changes and you get back two different results it’s hard to deal with. I’m fine with two different results if the wording in a response is different or something like that. That’s sort of expected and one of the weird things about these generative AI models. But to take action in two different ways creates a lot of problems.
Until hallucinations are better understood and easier to deal with on an application layer, I think using generative AI in industries like healthcare is going to be very difficult. You cannot use something that could change day-to-day in a field where people’s lives could be affected. And maybe this is why Apple was so restrictive and conservative about their review, because no one actually knows how these things truly work.
In the end, healthcare is just a really hard domain to navigate in, especially when trying to do new things. It makes sense why this is the case, but it also means that certain outcomes are just not possible, or they have to be done on the margins, with those willing to take a risk.
Weirdly enough, I think think make AI in healthcare a big opportunity. I’m sure Sentinel isn’t the only healthcare business that struggled to make AI work. In fact, I know there are many businesses that have tried deploying it in healthcare and have hit lots of different hurdles. But that in lies the opportunity. For anyone who can overcome these hurdles there’s a massive opportunity waiting.
How to make it work
I’ve now detailed all the mistakes that I made and the obstacles that I came across along the way to try and make Sentinel work. Obviously, I didn’t get to where I wanted to go. In fact, I came up far, far short of that vision. Perhaps I’m not the right person for it and/or the timing just isn’t quite right, but I do think the original vision is directionally correct and that the problem is still worth solving. For those interested in this topic then, the question remains, how could you make it work?
To start to answer that question, it’s worth restating the original vision for Sentinel and the problems it was trying to address. The original vision was that primary care was failing people because it’s still based on the model of one doctor for one patient and it’s there for you when you are not healthy but doesn’t do a whole lot to keep you healthy.
To tackle this, I thought the best approach was to attack it bottoms up. What I mean by this is that instead of tackling it head on and starting a new primary care practice (sort of like OneMedical did), I would instead start by picking off piece by piece each of the different services that a primary care doctor provided, eventually stitching together the full offering. I went about it in this way because I didn’t feel as if I could get the funding necessary to make the head on approach work, nor did I think strategically that it would be a good idea to do that.
Raise a good amount of money and build the right team
However, if I had to do it all over again, I think I might reconsider this approach. Not that it wasn’t a logical approach for all the reasons I just mentioned, but rather, healthcare startups face major hurdles when small. Small is a bit relative here so let me explain what I mean by it. When I say "small" in this context, I mean that I think you’ll set yourself up for the best chance of success starting a healthcare business if you go into it assuming you will need at least $1 million in seed funding to start with. To investors and in the grand scheme of healthcare, $1 million is small. But to someone who’s trying to bootstrap a business on their own to get it off the ground, $1 million is not small.
The reason I think you need to come to terms with at least starting with $1 million is that you need to be realistic about what your business is and what you’re going to have to do to make it work. With $1 million, you can do a couple things I wasn’t able to do bootstrapping. First, you can likely hire a very small initial team. I’d keep it to 2-3 people who are either savvy technically or in marketing or they have deep healthcare industry experience. The latter was a big missing piece for me. It’s likely why I was never going to be able to raise any money before having a product that already had customers and earning revenue. Having industry experience, and even better an existing network in the industry, could’ve unlocked multiple insights and avoided going down the wrong paths. Although it wouldn’t have to be a medical provider, having one would be invaluable. All of these initial team members should be part of your founding team, giving them significant portions of equity to justify lower than market salaries.
The next thing a decent amount of money in the bank will help you with is in setting up all the different services the offering needs to have off the bat. Instead of trying to go piece by piece and do lab tests first, then offer prescription management, then consultations, etc. I think it would be better to have it all at launch. I know this seems to sound like it’s counter to the lean startup advice, but again, what is the "minimal" thing here to test your core hypothesis. If you’re trying to say we think we could replace primary care with this new alternative, then you have to actually test that out. If you just do lab tests, you’re not testing that hypothesis, you’re testing whether or not people will use you for their lab tests.
Maybe I’m wrong here but I think that the target customer’s mindset is a 1:1 replacement. They are thinking about it like "I know I need a primary care physician, so why would I use a different lab testing service AND still have to get a primary care physician anyways". Offering all of the basics in new ways, will make that switching cost easier on your target customer. This is going to take longer to launch and there’s some risk around whether or not people will make the switch, but you at least know there’s a market that exists for this service.
The last thing that I think you have to realize to make a consumer healthcare product work is that in the end, your best asset is going to be your marketing. Because in medicine you cannot really vary what you do all that much (see Challenges with AI in Health above). Your true differentiation on the business side is going to be in your brand and marketing. Knowing this going in, you might approach things differently then I had17. This doesn’t mean that there isn’t room for technical innovation here but that you’re primary advantage will most likely not lie in the tech, it will be the brand. If this isn’t your cup of tea, then start a B2B healthcare company instead.
Target a specific condition
Outside of the above approach, I think there are two other approaches you could take with the product to potentially make it work. The first would be to make your target customer market patients suffering from a particular chronic condition. There’s a couple of advantages to this approach. The first is that it helps narrow focus, which is good for startups and good for product development. It’s easier to develop features with a narrow customer market. If you’re targeting people who suffer from diabetes, for example, you can focus on adding features around the management of insulin and glucose monitoring, vs trying to provide just generalized features for everyone. The second advantage to this approach is that the target customer is highly motivated and therefore has a significant need. Chronic conditions have the unfortunate nature of typically having no cure and therefore a lifelong endeavor. By focusing on this subset of patients, you might be able to provide them with much more than just throwing drugs at them, which is the typical healthcare approach to these problems. Community support services, alternative care approaches, etc might be some of the things you could offer that others wouldn’t.
Combine care and health insurance
The other approach that I think could work but would be very difficult to pull off (a moonshot sort of approach) is to provide both care and insurance as one service. This was actually the seed of an idea that led to Sentinel. I originally viewed the problem with healthcare boiled down to the misaligned incentives that exist. If doctors and hospitals get paid to treat, that’s what they’re going to do more of, treat. However, the counterbalance that existed was health insurance. Health insurers don’t want over treatment because they are often the ones footing the bill. But of course, everyone hates their health insurance provider, few hate their doctors. The thing I was interested in is combining the two sides to align incentives. If you both provided the insurance for care and were the caregiver yourself, you could better align incentives, while still having a business model (the revenue from the insurance premiums).
The reason I never took this approach with Sentinel is that there was no way in hell I could pull it off by myself bootstrapping it. Not that this was what I wanted to do, but there was also no way I could raise enough money to make this a reality. If I thought I wasn’t going to be able to even get $1 million to get an alternative primary care practice off the ground, then there was no way I’d be able to raise the $100+ million needed to get a health insurance company off the ground. But, in theory there are probably people who could do this. And if you combined it with the above idea of targeting patients with a certain condition, you could really create quite a unique offering. One that insured patients with a particular condition and provided them with the specific care that they required, focusing your product offerings and your insurance underwriting. I haven’t heard of any companies like this that exist and perhaps there’s a reason why, but it’s super interesting nonetheless and perhaps worth pursuing if you’re the right person.
Start smaller
For the most part, all of the above approaches require a venture-backed approach. Perhaps if you had the right audience and had enough money of your own, you could got at the above without venture money, but that’s only going to be true of a small select group of people.
If, however, you wanted to try and do something in consumer health that was bootstrapped, I think your only option is to start really small. This is probably good advice generally, even for those that know they are going down the venture-backed path, but it’s essential for bootstrapped companies.
This is what I tried to do with Sentinel, I just likely chose the wrong path. If I were to do it again, I’d try to avoid gatekeepers like Apple, create something that could work on the web and avoid building any sort of data integrations at first.
One approach might be to just build a web app that could help people get all of their lab work data in one place. Instead of leveraging Apple’s HealthKit Health Records SDK, you could just start with a simple document upload.
You’d likely have to make the first versions of this free, though I guess you could charge users a small amount to test if there’s a willingness to pay for this. From there, assuming you were able to generate some interest, you could start to layer more interesting things on top of it.
For example, a lot of recent papers in medicine are using lab tests for various findings. One can calculate a “biological age” based on only a subset of lab tests. The specific age doesn’t matter so much as the relevant distance you are to your current age compared to your peers. This seems to have some predictive capabilities on all-cause mortality.
Another paper shows that current blood test reference ranges aren’t that valuable to an individual. What is, is their own personal reference range. These personal reference ranges could then be used to be predictive of potential disease within healthy individuals. With enough data, you could offer your users this capability.
And you can continue to go from there, offering additional insights and services on top. Will this work and avoid some of the issues and challenges I’ve highlighted above? Not sure, I’d expect you will eventually run into many of the challenges I’ve already highlighted. But at least with this attempt, you can initially avoid the gatekeeper issue and at least launch the product to get feedback on it.
It sucks that I wasn’t able to get Sentinel to work because I think a lot of people can agree that the current healthcare system is broken and there needs to be a better way. Hopefully sharing here what I think could work if I were to try to do it all over again is helpful for others. And for anyone who ever wants to talk about this stuff further, drop a comment18.
What’s next
So, what’s next? Besides licking my wounds, I’m trying to take a step back and re-evaluate. The healthcare space is a large one with lots of problems to solve and I already have a company to bring a product to market, but for all the reasons I’ve outlined above, I’m not about to rush into it again. I’ll likely try and take another crack at making Sentinel work in another way, perhaps with one of the approaches I just shared. We’ll see.
For now, I’m just going to continue to build things, tinker around with new ideas and see if anything starts to pull me in. Until then, I’ll try to share more here about the things I’m thinking about and exploring.
✌️
Yes, creating a super intelligence has obvious positive business applications in ways any technology off the shelf does not.
There’s an inherent interesting conflict that exists between society needing people to take big risks and the challenge to an individual of taking those risks on. Society can avoid having a high rate of failure, as long as you have at least some big wins (this also happens to be the business model for VCs, it’s worth noting). Individuals, however, are extremely limited in their lifetimes. You might get a few shots on net in your life, but you’re likely not going to get more than 10s of shots on net. That’s why the rewards for doing this when you do succeed need to be so large. Something I’m realizing more (even though it seems obvious in retrospect) is that being pulled into something gives you a higher chance of success than pushing something onto others.
This is another way of saying whether or not you have product/market fit. The successful businesses had it, the unsuccessful ones did not. The debate here is a chicken and the egg style debate. Should you start a business first before product/market fit? Or should you get product/market fit (or at least some signal for it) before starting the business? I think it depends. In the case of bootstrapping, it probably makes more sense to be pulled into it. If you’re going after a big idea that needs funding and would be huge if it works, you have more freedom to figure it out as you go. I’ll leave it up to you in which half your idea falls into.
Unless you’re loaded.
You can argue whether or not this is a positive thing but I think it generally plays out this way. Venture backed startups eventually run out of money and shut down if they never find product/market fit but depending on the amount of capital raised, this can go on for a really long time. Bootstrapped businesses don’t typically have the ability to find product/market fit without some sort of money coming in to sustain the search.
One additional piece of advice that might help here is having cofounders. I could have listed not having cofounders as a mistake but my plan was to get the business to a good spot before trying to recruit cofounders (it’s a lot easier with funding and some hint of success). You could argue it was a mistake to go at it alone until that point, but regardless, cofounders can make the struggle suck less. That said, there are also downsides to cofounders too, so it’s a double edged sword.
An example of something that seems small/not like work but can be super valuable is actually reading and posting on Reddit. I’ve under appreciated the treasure trove of information that Reddit provides for so many different types of communities. It doesn’t feel like work because it’s doing something that we just do for entertainment but the insights can be very valuable. I’ve also found it easier to get people to talk about things there than paying them for customer interviews. The caveat with this is that it can be inefficient, since you kind of have to dig for the value. Also, some communities there are better than others, so it all depends on your idea and the communities around it. That said, it’s an underutilized tool for market research IMO.
I’ve written about the importance of distribution in business and software products before. You can read about that as it applies to ChatGPT plugins (now GPTs) here.
In hindsight, I probably could’ve saved money by spending more here. Without getting into the nitty details, my ad spend for this was in the $100s, whereas, other services and such that I paid for on the product side, reached into the $1,000s in the end.
This likely is a lesson to learn here too. Because it worked for me in the past, doesn’t mean it’d work for me again. Not only that, but I likely mis-valued it as a result of that bias.
Because the only way to make a low price business work is with a higher volume of customers and if you can’t get high volume, you can’t charge low prices.
Another fun fact that I learned that’s related to insurance companies and healthcare oddities is that your electronic health record (aka your eHR) is 1) not actually owned by you (it’s owned by the provider who created it) and 2) is designed for billing purposes, not for health data record keeping 🙃.
The healthcare system really is an upside down world but you can sort of follow how it’s become this way by following logic and incentives. It was just poorly thought out and incentives are all misaligned. This actually is one of the things where AI in health could have a big impact. Making it easier and less time consuming to deal with all of the complexity and crap that comes with dealing with health insurance could bring costs down across the board.
Another feature of the upside down world of health is that sometimes charging cash for services actually makes them less expensive then they would be with insurance, including after insurance pays for some of it!?
Of course, many will do more for you if you throw a bunch of money at them.
The idea here was to use a combination of AI and doctor, with the hope being that the AI would reduce the need for the human doctor, except for particular instances. However, with the way pricing plays out for these different services, having less doctor consultations and interactions actually is worse because the low volume makes it worse on their end.
This is mostly due to financial shortcomings, being bootstrapped and all, rather than it being difficult to implement.
As a side note. I have previously mentioned Mark Cuban’s CostPlusDrugs.com as an example of one of these new consumer healthcare companies. I recently listened to a podcast where he is asked the question of whether he had to be a billionaire to start the company, to which he says yes. I’ll challenge this. It’s not that he had to be a billionaire to start the company. As I’ve said, it might take money to get it going, but it doesn’t take a billion dollars. Instead, the reason the company was able to get started and be as successful as it is today is because of his celebrity. Look at how they use his name every where. It’s not CostPlusDrugs.com but "Mark Cuban’s CostPlusDrugs.com". Maybe he had to be a billionaire to become famous but it’s the fame that helps, not the money.