swagger-gist.io – Saving and Versioning Swagger definitions

TL;DR – Check out swagger-gist.io to start saving Swagger definitions as Gists.

During my time as an API Product Manager, I found myself constantly looking for ways to make my life easier and more efficient.

One of the problems that I encountered was that it was difficult to share and accurately version my swagger definitions.

I started using SwaggerHub, but I found it to be overkill for the simple Swagger definitions that I was creating.

Eventually, the easiest way for me was to use Github’s gist service frequently to push YAML or JSON files and was asking people to copy these into editor.swagger.io to view and manage.

Given that this relationship was quite nice, I thought I could modify the open source Swagger editor to auto save into Gist for me.  This both allows me to continue to share my Gists, whilst removing the need to manually save them there.

After a bit of tweaking, and a bunch of AWS services, I’ve today pushed live swagger-gist.io.

Get Started:

  1. You’ll need a github account, so sign up at github.com.
  2. Follow the instructions here to generate an access token. Please make sure you grant Gist access when creating the token.
  3. Browse to swagger-gist.io and click ‘File -> Connect to Github’
  4. Enter your github username and the access token that you generated.
  5. Click ‘File -> Open API Definition’ and this will search your Gists for any that have a ‘Description’ with the word ‘Swagger’ in it.
  6. Click ‘Open’ and the app will retrieve the JSON gist and render it as YAML.

Note: i suggest saving this access token in your favourite password safe as you may need it across multiple devices.  At this stage I only save it into the browser’s localstorage object.

Creating a new Swagger Definition

If you don’t have any Swagger as Gist yet, then you’ll need to create a one using the instructions below.

For the moment you have to create new definitions manually, but once it’s created, the auto-saving will take place. I’ll improve this eventually.

  1. Browse to gist.github.com and sign in.
  2. In the gist description, make sure you include the word ‘Swagger’ (with the capital S) and make a note of it.
  3. The filename can be whatever you desire, just make a note of it.
  4. Enter some sample value in the gist (e.g. ABC123) it doesn’t matter what it is.
  5. Click Create Public Gist.
  6. In the URL you will now see something like: https://gist.github.com/johnsmith123/40eea519f479fe27a831d15770255a7f
  7. The string of characters after your username is the Gist-ID.  You’ll need this next.

Creating the new file in Swagger-Gist:

  1. Click ‘File -> New’
  2. You’ll see in the sample file ‘info’ section, there are 4 elements that should be filled out:- title
    – x-gist-id
    – x-gist-filename
    – x-gist-description
  3. The title can be whatever you desire, but the id, filename, and description should match exactly what you created in your sample Gist.
  4. Once you’ve copied the properties, the auto-save will automatically occur after 10s.

That’s it! You’ve now got a swagger definition that is hooked up to Gist.

Next Steps:

  • Hook up proper Github OAuth to generate the token and store them in a more permanent store.
  • Allow generation of new Swagger docs directly from the editor
  • Publish to Heroku (I use Heroku heavily for hosting swagger definitions).
  • Migrate code to the latest fork of swagger editor.  It’s changed significantly since I started this project.

Tech Tools Used:

Front End: Angular JS
Hosting: AWS S3 / CloudFront

Back End: AWS API Gateway -> Lambda -> Gist API.

Shout out to @gojkoadzic who created claudiajs which helped me significantly with writing my AWS API / Lambda code.

If you’re interested in finding out more, ping me or comment and I’ll write another post.

API Management Platforms – do I need one?

One thing every enterprise API creator needs to think about at some point prior to the API going to production is whether an API Management Platform is required.

The main ones that I looked at when we were in the design phase of our API were:

  1. Mashery (http://www.mashery.com/)
  2. Apigee (https://apigee.com/)
  3. 3 Scale (http://www.3scale.net/)
  4. Layer 7 (http://www.layer7tech.com/)

There are lots of others out there, but these four seem be the most active contributors to the API Management community.

What exactly is an API Management Platform?

All of these platforms all fundamentally do the same thing, but each have their nuances and provide different features and functions.

Generally they act as a proxy between your API and the customer, consuming application, or partner.

 

These platforms protect your back end services by limiting the customer to a certain number of allowed queries per second (QPS) or queries per day (QPD).

This prevents a user from inadvertently performing a DoS through poor coding or some form of attack.  If they can only get through a small number of QPS, they’ll bring down the API Management Platform before they’ll bring down your service.

Along with this, they also generally provide:

  • API Key & Authorisation Management
  • Usage Reporting and Analytics
  • Live documentation (Swagger/io-docs)
  • Developer Community Management
  • Billing and Payment Management

Why not just build it myself using mod_qos?

Our network administrator was adamant (and still is) that he could build all of this for me with without the need to pay an external vendor.

Projects like Apache’s mod_qos provide an excellent solution for throughput management, this bundled with an analytics engine (awstats) and you’ve got half the service done right?

Wrong. Do you really feel like designing all of the features and functions specifying exactly what you need.  How long is that going to take with all of the other stuff your network guy (or girl) has to work on? What about ongoing support, maintenance, new features… don’t you have an API programme to be worrying about?

The beauty of these SaaS/PaaS providers is that they’ve done the thinking for you, and they are generally experts in the area, so they know what you want before you do!

They also generally have great Partner and API Growth programmes, so if you can get your API on their marketing ticket, then you’re kickstarting your growth before you’ve even lifted a finger.

We had web demo’s from both Mashery and Apigee, and in the end we ran with Mashery.

Implementing an API Management Platform

The main reasons we chose Mashery was that it seemed very simple and easy to use, their price point was right for us, they had all of the features out of the box that we were looking to implement, and they could stand up an environment for us quickly.

They’ve got some really smart people and they will give you some valuable insights into usage patterns, similar use cases and implementation best practices.

When implementing an API management platform, you’ll also need to think about:

  1. The UI of your developer portal; they can help you with implementation, but the design is up to you
  2. Sandboxing/Trial access for your developers and the associated registration flow
  3. Documentation of your API; are you just going to use swagger/io-docs or are you going to manually write the documentation.
  4. Network level access; whitelisting/blacklisting and firewalls.

Downsides/Pitfalls of using an API Management Platform

  • Change Requests, customisations can be difficult to get approved, and can be costly
  • VSMT isn’t really supported, so stick to generic content types
  • It can be harder to debug issues (timeouts, network access, authentication)
  • Their downtime (if any) may reflect poorly on your API

In the end, it really does come down to what will work in your organisation.  In my opinion, I believe there is value in using an API management platform.

Their whole purpose is to assist your company in building out your APIs, so it’s in their best interests to help you grow.

It will assist you with;

  • Speed to market
  • Accordance with best practices
  • Out of the box reporting, key management and developer portal/CMS
  • Network opportunities with other organisations

However, this may come at an up-front cost, and a bit more effort to get things going.

Happy to answer any questions, or even introduce you to my good friend Jason (from Mashery). Just leave a comment.

Jord

Designing the API – Identifying the resource types

Before I jump directly into my version of “Donny Do” and “Donny Don’t” for APIs, I’ll give a bit of background as to the steps I took to develop our API.

(Note: If you get REST and how to structure a REST API, skip this post)…

When I was asked to design our API my first step was to watch Teach a Dog to REST. Then watch it again, and again. I made other members of the team watch it, and took as much as I could from it.  This video is probably the most elegant demonstration I’ve seen that articulates “proper” REST API design.

From here I took my instructions from the video and broke down the application into its different types of resources.

As the application I was dealing with was to do with SMS messaging, the resources became things like messages, message templates, recipients, and recipient groups.

I didn’t start with the use cases, e.g. “how do I send a message” because identifying the resources and the resource types within the API is far more important at this stage than worrying about what you can do with the API.

The reason is that if you focus on the use cases (the verbs) you’ll end up with a list of actions. This may sound OK, but ultimately this path will lead to a bunch of action based URLs e.g. /sendMessage, /createRecipient or (worse) /addRecipientToDistributionList.

Focus on the resources (the nouns) instead.

What you actually want at this stage is simply the list of resource group URLs:

  • /messages
  • /templates
  • /recipients
  • /recipientgroups

From here I simply thought of each of the main 4 HTTP verbs acting on my URLs; POST, GET, PUT and DELETE (Create, Read, Update and Delete respectively).

Resource GET POST PUT DELETE
messages Retrieve a listing of messages. Create and send a new message Not Implemented Not Implemented (could refer to deleting all messages)
message templates Retrieve a listing of message templates Create a new message template Not Implemented Not Implemented (could refer to deleting all templates)
recipients Retrieve a listing of recipients. Create a new recipient Not Implemented Not Implemented (could refer to deleting all recipients)
recipient groups Retrieve a listing of recipient groups. Create a new recipient group Not Implemented Not Implemented (could refer to deleting all groups)

Note: we still haven’t gotten into any schemas or technical implementation details.  During this phase I am trying to define a simple API structure; that’s it.

After completing the table for the resource groupings, I then moved on to each individual resource.

Resource GET POST PUT DELETE
messages/[ID] Retrieve an individual message Not Implemented Update an individual message (possibly relevant for drafting) Delete an individual message
messagetemplates/[ID] Retrieve a message template Not Implemented Update an individual message template Delete an individual message template
recipients/[ID] Retrieve a recipient record Not Implemented Update an individual recipient record Delete an individual recipient record
recipientgroups/[ID] Retrieve a recipient group Not Implemented Update a recipient group’s details, or add a recipient to the group. Delete a recipient group

Structuring my table like this forced me to think about what each action is going to perform as I only had 4 actions to work with (there are a few others but we’ll get to them later).

For example, I made the call that when you performed a POST to /messages this would not only Create the Message, it would also Send it.

Some would argue that the resource should have been created in one step, and sent in another e.g. create it with a POST to /messages, then update it with a PUT to move it to a ‘sent’ state.

This was an option I considered, but I decided that more of my users would be sending messages in one single step than there were those that would require a draft/send process, so I went with the two in one approach.

When you are designing your API, I would encourage you to start here also.  It will take some time to map out what each resource is, and what the 4 verbs mean for each, but it will be worthwhile in the long run.

Building An API – Lessons Learned

In 2011 my boss came to me and asked me to design an API for our product. I had been exposed to APIs in the past in my role as a developer, but I had never thought about designs, architecture or best practices.

After doing a bit of research, I presented my team with a macro design about how the API would work.  It was going to follow this great standard for APIs called ‘REST’.

The following links were invaluable during the concept and design phases of the API:

RESTClient
The Firefox Rest Client plugin (a MUST for any API work).  The Chrome ‘Advanced Rest Client’ is also extremely useful.

Best Practices for designing REST APIs
A really excellent article around the reasons why REST APIs should be designed the way they are. Brilliant stuff.

RESTful API Design: Teach a Dog to REST
A great presentation on the best practices and principles of restful architecture and design.

REST API Tutorial
A step-by-step tutorial on what REST is and why it is good

REST Anti-patterns
A good article about the do not’s of REST

How I explained REST to my Wife
A condescending look at how to explain REST to a non-technical person.  I thought this was a really good article about why REST should be used correctly for machine-to-machine APIs.


The API that I designed has now been in production for about 3 years and I’m writing this post because if I was doing this all over again, there’s a few things that I would have liked to know; namely:

  1. REST – Best Practice vs Practicality
  2. Practicalities of versioning your API
  3. How do users want to see you securing your API
  4. API Service Providers (Mashery vs Apigee)
  5. API Deployment and Updates
  6. Developer Community & Examples
  7. Carding for Developers (Agile Story Cards)
  8. Documentation (i/o docs & swagger)
  9. Gaining user feedback on the API

Over the next few weeks I’ll be populating this blog with content around these areas and others.

Please feel free to drop me a line if you’re curious about any other aspects of the API design and current implementation.

Cheers

Jordan