Generating API Docs. Is it possible? Part 3

Generating API Docs. Is it possible? Part 3

Turns out, it is!

In the last blog post I wrote about using json api resources to generate an openAPI document for all endpoints and its components.

It's a nearly complete description on the api, except for a couple key component which require a bit more manual handling.

The approach I decided on was to add comments to each function and class that required it and use RDoc as a comment parser to add to the existing API documentation. RDoc has the option to parse individual files and attach a comment to a specific class, function or attribute. Super helpful in our case determining what function is returning what type of value, describing what it does in more detail, or even hiding them from the api in case of an internal value that the public does not need to know about.

Using RDoc on specific files

Ironically the documentation on parsing individual files in RDoc is not very clear and took me a while to figure out while being relatively new to ruby. What I've come up with is the the snippet below. The snippet uses a filename and uses RDoc to process the file and setup its own internal structure with attributes, or methods and its corresponding comments.

filename = '/somerubyclass.rb'
@store = RDoc::Store.new
@options = RDoc::Options.new
@stats = RDoc::Stats.new @store, 1
content = RDoc::Encoding.read_file filename, 'utf-8'
top_level = @store.add_file filename
parser = RDoc::Parser.for top_level, filename, content, @options, @stats
scan_result = parser.scan

# comments related to the class
unless scan_result.comment.empty?
  parsed_comment = scan_result.comment.parse
  ...
end

# comments related to attributes and methods
scan_result.classes_or_modules.last.method_list.each do |m|
  parsed_comment = m.comment.parse
   ...
end

After running this for our ruby class, we'll have a hash of attributes and methods with a value of its corresponding comments. The comments that are returned are in a list of lines. Unfortunately the comments themselves need to be parsed manually to determine whether it's a return type in the comments, or a description or something else.

We can write a simple parser for that and check what work the line starts with. Here are a couple examples of a relevant comment for our parser:

  ##
  # The total amount of logins for this user
  # @return [Integer]
  def login_count
    ...
  end

  ##
  # @hide
  class Api::V1::HiddenResource < Api::V1::BaseResource
     ...
  end

  ##
  # This class is used for all kinds of currency and can be used for...
  # @tags Payments Checkout
  class Api::V1::CurrencyResource < Api::V1::BaseResource
     ...
  end

In this example, you can see that the comment starts with ##. This is required by RDoc as a standard and marks the beginning of a comment that is to be parsed. The line without anything can be anything. Here I'm interpreting it as a description of the method. And @return is the return type of the method. Since we're writing our own parser, I've added a couple custom attributes that were helpful for my documentation

  • @hide don't display this at all (a class, attribute or method)
  • @tags a list of tags i want the class to be grouped under in the document

When you have the parsed comment you can look at it line by line and create a hash of your decorators like so:

def parse_comment_lines(comment)
    return unless comment.parts.count.positive?

    # parse comment line manually since there seems to be no option to do anything else to this comment
    comment_map = {}

    comment.parts[0].parts.each do |comment_line|
        if comment_line.downcase.start_with?('@return')
            comment_map['returns'] = comment_line.downcase.split('@return').last.tr('[] ', '')
        elsif comment_line.downcase.start_with?('@tags')
            comment_map['tags'] = comment_line.split('@tags').last.tr('[]', '').strip
        elsif comment_line.downcase.start_with?('@hide')
            comment_map['hide'] = true
        else
            if comment_map['description'].blank?
                comment_map['description'] = comment_line
            else
                comment_map['description'] += comment_line
            end
        end
    end
    comment_map
end

Putting it all together

With using the following items we now have a almost complete picture of how the api should behave that all can be generated on the fly:

  • All paths based on the resource classes
  • All attributes for each component (that is used in a path) and its types based on the model schema and return types in the comments
  • Adding descriptions, grouping and hiding paths or attributes

With these blog posts you'll have dots you can now connect with some more code, since all the json returned based on the json api spec is very predictable.

The only thing that is missing at this point are the scopes that a request can have. And those are tricky to programmatically define in an openAPI schema. A scope can be an admin user vs a regular user. For example the admin will see more fields than the regular user. This is often based on the auth token and the role that is described within that token. Since I don't know how to programmatically access that in way to list the possible scopes and differences, I opted to list details like that in the description of the resources and keeping the schema simple and straight forward.

In short, this is what the core of the api doc generator does:

def generate_api_docs
    resources = ObjectSpace.each_object(Class).select { |klass| klass < JSONAPI::Resource }

    resources.each do |r|
      single_name = r.name.demodulize.gsub('Resource', '')
      slug = single_name.underscore.gsub('_', '-').pluralize(2)

      model = single_name.safe_constantize
      model_decorations = parse_comments(model) unless model.nil?
      resource_decorations = parse_comments(r)

      # if there is a comment and there its set to hide, skip this resource
      unless resource_decorations[r.name].nil?
        next if resource_decorations[r.name]['hide'] == true
      end

      paths["/#{slug}"] = {
        get: generate_get_all_path(r, single_name, resource_decorations),
        post: generate_post_path(r, single_name, resource_decorations)
      }

      paths["/#{slug}/{#{single_name}Id}"] = {
        get: generate_get_one_path(r, single_name, resource_decorations),
        patch: generate_patch_path(r, single_name, resource_decorations),
        delete: generate_delete_path(r, single_name, resource_decorations)
      }

      schemas[single_name] = generate_definition(r, single_name, resource_decorations, model_decorations)
    end

I hope you enjoyed reading about this, as much as I was about figuring out this puzzle of keeping our API docs up to date without having to spend a lot of time to maintain it!