5th May, 2015
Data has become part of the fabric of applications: in the past 18-24 months, we’ve seen developers use data and analytics as part one of their frequently used tools, alongside more traditional endeavors such as working on front-ends, mobile and backend operations. Developers are using data to get a better look into their applications, their operations and the customers, not just retrospectively, not just in real time, but to make predictions on the future.
It’s to that end that we introduced Amazon Machine Learning at the AWS Summit last week, a fully managed, predictive analytics service, geared for developers. I was lucky enough to be part of the announcement at the event, so I thought I would share a few more details on the service here, my freshly minted new blog.
Machine Learning and Amazon
Machine learning is an approach to predictive modeling: the ability to automatically find patterns in existing data, and use them to make confident predictions on new data. For example… based on what you know about your customer, you can ask the question: “will they use our product?”; based on what you know about an e-commerce order, you can ask “is this order fraudulent?”, or based on what you know about a news article, you can ask “what other articles are interesting?”. Machine learning lets you use existing data to build predictive models to answer all these questions, confidently.
At Amazon, we’ve got a deep history in using machine learning: for example, even on very early gateway pages from amazon.com, you could see evidence that we were using machine learning to automatically make recommendations for customers. And since then, we’ve used it across many areas of the business, including on product detail pages, with the famous “customers who bought this also bought” feature. We also use machine learning to improve search, to understand and process speech and natural language (for a recent example, look no further than Amazon Echo), or in improving our fulfillment operations with new vision systems that enabled the unloading and receipt of an entire trailer of inventory in as little as 30 minutes (instead of hours).
And from these examples, and in talking to our customers, we’ve seen that the key challenge of machine learning is that, while there is some overlap in the expertise required with the typical SDEs, it’s sufficiently small that there is a huge amount of friction for developers who work with their data day in, day out, to apply machine learning to that data. Instead of building and reusing software components, there is deep expertise required in statistics, model building, validation of those models, algorithm selection and optimization and data transformation.
The friction gets even larger when you need to put these predictive machine learning models into production, so that they are optimized, and high performance, and even further when you need to do this at scale: scale of both the volume of data needed to build and validate predictive models, but also in putting those models to work in high traffic mobile apps or web sites.
What we’ve heard from customers is that they see a large opportunity of using existing and growing datasets with machine learning, but that to take advantage of this, there has to be a very low barrier to entry
Amazon and Machine Learning
This was the driving force behind Amazon Machine Learning, a fully managed machine learning service, geared and tooled for developers - available today in your AWS console. The service exists to allow any developer to apply predictive modeling to their data in minutes, not months, and to provide easy to use tools which help get the best from those models. We saw wonderful things when we put machine learning in the hands of the development teams at Amazon, and I’m looking forward to seeing the same level of innovation across our customers at AWS.