How machine learning changes fraud detection - with Jason Tan of Sift Science

When he was in college, Jason Tan thought he was headed to Google or Amazon. Early career setbacks were the push Tan needed to embrace the startup life. He's now the co-founder and CEO of Sift Science, a growing fraud detection startup grounded in machine learning.

Recently, I talked with Tan about how career adversity led him to "embrace the weird," and how being unapologetically different became a core part of Sift Science's approach. We also talked about the art of fraud detection, and why machine learning opens up new ways for customers to guard against sophisticated online fraudsters.

Jon Reed: Tell us where Sift Science stands now.

Jason Tan: We've come a long way since Sift Science was founded in June of 2011. We went live in March of 2013 with a publicly-available product. In all, we've raised three rounds of funding, a seed round, a Series A, and a Series B, totaling roughly $24 million. Our seed round was led by Max Levchin, who was the co-founder of PayPal.

Reed: You can't look more than ten feet without stumbling on a company bragging about AI or the Internet of Things. What accounts for Sift Science's success amidst that noise?

Tan: We focused on solving a very real problem. Your point about the hype around machine learning - I think there are some legs to that hype. But there's also a lot of generalized statements about machine learning that don't lead to a result. Realizing the potential and applying it to a concrete problem - that's the value. When customers look at your website, they need to be able to understand your value proposition in five seconds or less.

Reed: You've come a long way for someone who wasn't planning to be a startup guy.

Tan: The answer to that fits well with your theme of embracing the weird. Back in college, I was very anti-weird. When I was a college senior, I was brainwashed to believe I had to go to a Google or Amazon or Microsoft after I graduated, I was singularly obsessive-focused on getting a job at one of those three big companies - because that was the mainstream choice.

The funny thing is, I interviewed with all three companies and I did not get an offer with any of them. Software engineering interviews tend to be very problem-solving oriented, so you have to write code on a white board with someone over your shoulder, and it's a bit unnatural. In that kind of environment, I actually get very nervous, and so when I was interviewing with Amazon, Microsoft and Google, I just got really nervous. I bombed at those interviews, so I didn't get an offer from any of those companies.

Reed: That must have led to some dark days.

Tan: It was pretty heartbreaking. I had this dream and I didn't achieve it. But looking back, that was probably the best thing that ever happened to me. My friend who had interned at Zillow the summer before put my name in with the VP of Engineering, and that VP of Engineering reached out, saying, "Hey, we heard good things, we'd love to talk." I said to myself, "I don't know anything about startups, I don't even know what Zillow is, but I have nothing to lose."

Reed: And?

Tan: Zillow is now a publicly-traded company, and they're one of our customers. The CEO and co-founder of Zillow is an investor in Sift Science today. Our CTO at Sift was my first manager, Fred Sadaghiani. All these good things came out of Zillow.

Reed: Plus you caught the startup bug.

Tan: I learned that in a startup, you can have an impact that's hard to match at a big company. In a startup, everyone needs to punch above their weight class. Everyone needs to chip in. If you're not contributing, it's not the right environment for you. You can learn so many different things at a startup. I think that's so important when you're getting started in your career. I also think startups have more of a meritocracy, where if you can prove yourself and do good work, you're going to get promoted more quickly.

What does machine learning have to offer fraud detection?

Reed: Tell us about the start of Sift Science.

Tan: In 2011, I moved to San Francisco to do Y Combinator with my co-founder Brandon Ballinger. At a high level, we're trying to empower online businesses with large-scale, real-time machine learning so they can accurately predict risk in real time, and be able to deal with their growing volume of transactions in a scaleable and efficient way.

Reed: I thought fraud detection systems had been around for a while.

Tan: True - but the traditional way of solving this problem is with rules-based systems that don't adapt and don't learn. Imagine if your email spam filter required you to set up a bunch of rules to filter your spam, so for each spammy email, you had to set up a rule. If the subject line has the word, "Viagra," or, "weight loss," you'd have to set up a filter for that word first. That would not work in this day and age.

Reed: When you say "online fraud," are you primarily focusing on credit card fraud and payment scams?

Tan: For us, fraud is not just stolen credit cards - it's any type of bad behaviors. People abusing referral programs, people posting spammy content, people doing account takeovers. There's a lot of different types of bad behaviors out there. A machine learning system is going to help these online businesses proactively identify those behaviors before they happen, and not put a huge burden on those businesses to set up rules-based systems and do large amounts of manual reviews.

Reed: So how do you teach a machine to get smarter at identifying fraud?

Tan: I liken it to being an online detective, where you're trying to look for different clues. Often, it's not any one clue that is the surefire smoking gun. It's really a combination of subtle clues, like the number of digits in an email address. We've learned from our data that the more numbers you have in your email address, the more likely you are to be fraudulent.

We've also learned to factor in the distance between the billing and shipping address. If the distance between them is pretty large, that is a potential fraud indicator. We've also learned that if you don't capitalize your first and last name, you're more likely to be involved in fraud. We analyze thousands and thousands of these clues, piecing together combinations that paint a picture of what's happening.

Reed: But you also have to show that data to customers in a useful and actionable way.

Tan: Right. They need to understand whether you can help them with something. Machine learning is not the end result, it's the means to an end. You have to be able to translate your technology into something they can relate to. We don't bang the table that "We do machine learning." It is part of our messaging, but we try to make things more accessible. Our customer base may not be as technical as we are, and that's okay.

Digging into a fraud detection use case

Reed: Give us a customer example.

Tan: One example is an online booking system. They discovered us two years ago. They acquired an online gift card startup, and that company had heard about us through Hacker News, I think. We had done a technical blog post that went viral. Anyhow, gift cards are notorious for fraud. You're effectively selling a form of digital cash. You can use a stolen credit card to buy a $100 gift card, and you can use that $100 gift card to purchase something. That makes it very lucrative for fraudsters.

This gift card startup was seeing a lot of fraud. They needed a solution that could scale with their growth and do so in a real-time, automated fashion, because with the digital gift card, there is the customer expectation that a digital gift card should be delivered quickly.

Reed: And what was their experience?

Tan: We were able to help them scale efficiently without a lot of false positives. To this day, we are a very powerful driver of growth for their digital gift card program. Traditional ways of preventing fraud cast too wide a net, where you're flagging too many customers for manual review. That's inefficient, and it's not a great experience for those false positives that get flagged for manual review. If you can do that flagging accurately, you're going to be able to improve the experience for good customers, but also still catch fraud.

Reed: And when did they go live?

Tan: That was just under two years ago.

The pesky ROI question - measuring fraud detection results

Reed: So how do you assess the results of your projects?

Tan: We use the standard metrics of success that any fraud program would use. One is the false-positive rate. Out of the users you think are suspicious, what percentage of those are actually not suspicious? Those are good customers that might have been insulted in some way. Then there's the inverse of that, which is how much fraud do you allow through. That might manifest itself in terms of charge back rate, like what percentage of your orders are resulting in a charge back.

Or it might be: what percentage of content on your site is spammy content? How much bad behavior are you letting through? Then there's the labor expense of manual fraud review. What's the percentage of activity that you need to have a human look at in your review? How efficient is your review process?

Reed: And what kind of results are you seeing?

Tan: We've been excited to see that our solution is often an order of magnitude better than what's out there in terms of results. Take the metric, "What is the false positive rate on manual review?" The industry average is less than 20 percent, which means that as a fraud analyst, I'm spending 80 percent of my day looking at orders that don't need to be looked at, because they're good customers. That's a lot of inefficiency.

With Sift Science, it's actually the opposite. We're able to turn that around from an 80 percent false positive rate to a less-than-10 percent false positive rate. On top of that, we're able to drastically reduce the number of orders that need to be manually reviewed, because we're able to empower the business with automated decision making on everything else.

Reed: And how many employees you guys have now?

Tan: We have about 47 full-time employees, and thousands of customers around the world.

Reed: Sounds like your momentum is growing.

Tan: We take it one day at a time. I'm proud of what we've accomplished so far, but the journey is just starting.

End note: Tan had more to say on how embracing the weird can impact product differentiation, from marketing to pricing transparency. I'll share that in a later update.

Image credit: Internet fraud © Gajus - Fotolia.com

Disclosure: Diginomica has no financial ties to Sift Science. I was approached by their publicist and I found the pitch interesting.