This post is part of our ReadWriteCloud channel, which is dedicated to covering virtualization and cloud computing. The channel is sponsored by Intel and VMware. How one services provider provides first-class service to its agency clients. Learn more in this ReadWriteWeb special report, made possible by Intel and VMware: Opus: A Marketing Agency That Gets the Most out of Virtualization.
It now has more than 65 million tweets per day, most coming from services that use the Twitter API.
Twitter has made numerous changes to fix its API. Those experiences have taught providers what mistakes not to make when launching a service.
But there is still a lot for providers to learn.
Considering this, we asked developers and service providers to help us prepare a list of 10 common mistakes made by API providers. Hopefully, the list will provide some basic insights into what mistakes should be avoided when developing an API.
Our group of commentators include Adam DuVander executive editor at Programmable Web; Mike Pearce, a developer out of the United Kingdom who writes a lot about scrum and Agile; Mashery’s Clay Loveless and Sonoa Systems Sam Ramji.
1. Assuming Everything Will Always Work
“Databases fail, backend dependencies get slow, and/or someone somewhere along the line doesn’t escape output properly. The result? A nasty stack trace at the top of your API payload, which in some cases may even include production database credentials.Solution: Before you launch, kick over some dependency and see what your API does. Developers would rather have a predictable (valid XML/JSON) error message they can code for than a surprise invalid payload.”
Source: Clay Loveless
2. Poor Community Management
“Sometimes we see providers expecting the API alone to attract developers. You need a good service and must communicate to developers what it is you provide. Also, along similar lines, it’s easy to ignore the developer community management, but that communication can make or break your platform. When you get developers’ attention, keep it by being responsive and helpful. Treat them like partners.”Source: Adam DuVander
3. Not Anticipating How API Business Processes Will Scale
“APIs are about bringing the scale of the Internet to bear on your business – but what if that scale actually happens? Will a manual review for user on-boarding scale if you surge in popularity? Will auditing and liability processes get in the way of enabling free access to a subset of your data? Governance is less important when the API is less used, but when transaction volume becomes significant, APIs become important to the finance and legal teams. Don’t let success lead to failure due to non-scalable processes.”Source: Sam Ramji
4. Putting the API Under Website Domain
“Case in point: twitter.com is now going to api.twitter.com for the API. The result is that they can scale the website and the API independently of each other as needed. I’ve never heard of anyone regretting the decision to split out API traffic from main website traffic, and nearly everyone to doesn’t do that at the beginning wishes they had _and_ endures a lengthy & painful process of migrating developers to the new API location.”‘Source: Clay Loveless
5. Lack of Real-World Testing
You can often tell the APIs that have had some real-world use before launching, and those that haven’t. For example, an API that lists _some_ information about an object, yet needs a secondary call to get MORE information about that object. If your API requires two calls to find out everything there is to know, you and your devs will hate you for it eventually.Build a couple of reference apps that might do something useful, even if trivial. You’ll quickly get a sense for how sane your API is when you run into those inefficient API call combinations. If you haven’t built your website on top of your API, consider doing that — or at least what it would be LIKE to do that. If your API couldn’t reproduce your website, your API needs fixing.
Source: Clay Loveless
6. Not Anticipating Bad Behavior
1)Tactical Bad Behavior
- Inefficient client code can call an API too frequently, causing an accidental DDoS attack
- APIs can be attacked through the JSON or XML data passed as parameters or attachments;
- Attack techniques can include SQL injection, XML bombs, and shell scripts, as well as the usual social engineering and spoofing attacks.
2) Bad behavior “at scale”
- When your service is small and you have a hundred users, you may have only two bad actors. These can be pruned manually using firewall rules based on IP address.
- When your service is scaling to hundreds of thousands of users, you may have thousands of bad actors. These are placing high levels of “fake” demand on the service and can no longer be pruned manually without adding more staff dedicated just to these issues.
- To mitigate this do you apply rate limits to everyone and reduce the value of the service to good users? Do you find a way to apply policies to naturally separate patterns of use?
The anti-pattern we’ve seen here is the failure to anticipate bad behavior against your service, and then to respond either with draconian limits on use, or with intermittent systems availability as a result of dealing with the bad demand.
Source: San Ramji
7. No Black Box Tests
“Many APIs have no testing at all, and those that do frequently have automated unit tests, etc. Very few have end-to-end black box tests that can be run surrounding releases. Your unit tests may pass, but if your latest push also included a web server config tweak, you want to have an end-to-end test suite to validate those changes.”Source: Clay Loveless
8. Not Recognizing the API as a Core Line of Business.
“The companies who succeed in the new web economy recognize that APIs are a core line of business and essential path to customers and partners. Successful media, retail and web companies often see over 50% of their traffic from their API – and companies must treat it as a product.”Source: SamRamji
9. Not Having Management on Board with the API
Like any new project, business and technology need to be lined up together including having understood and documented goals, metrics, and rewards. What higher level objectives are being met and how do these map into departmental goals for marketing, customer support, and engineering? How will these be reported to senior management and on what cadence will performance be reviewed?Source: SamRamji
10. Tunneling Errors
“The specific mistakes you see regularly are error tunnelling (sending back a “200OK” response code, even if the body of the response describes an error). GET/POST tunnelling (sending all requests through GET or POST and ignoring the other verbs (PUT, DELETE, etc). And content-types (not allowing the consumer to specify the content type they will accept). Most of the big players get it mostly right. Amazon, for example, sets a good example, whereas Flickr does not. But most API providers miss the more esoteric stuff like hypermedia is the engine of application state and ignoring caching. However, you can use an API without that stuff quite adequately. It’s the incorrect tunnelling that will hamstring you and your consumers.”Source: Mike Pearce