How to create a Ruby HTTP API client gem - Josh Brody How to create a Ruby HTTP API client gem | Josh Brody
Back

How to create a Ruby HTTP API client gem

Most API client gems are either too clever or not clever enough. You either get a DSL that fights you at every turn, or a thin wrapper that barely saves you from writing raw HTTP calls.

This post walks through building one from scratch—the kind that feels obvious once it exists but requires real decisions along the way.

Why not use an existing solution?

Fair question. RestClient and ActiveResource exist. So does just using Net::HTTP directly.

Here’s the thing: generic solutions optimize for generality. They can’t make assumptions about your API’s conventions, error formats, or authentication scheme. You end up writing adapter code anyway—code that lives awkwardly between “library” and “application.”

A purpose-built client gives you:

  • Method signatures that match your API’s domain language;
  • Response objects instead of hash soup;
  • Error handling tuned to your API’s actual error format;
  • Key transformations that make sense for your codebase

If you’re calling three endpoints once, use RestClient. If you’re integrating deeply with a service, build a client.

Design first, code second

Before writing anything, I sketch interfaces. This isn’t perfectionism—it’s cheaper to change a sketch than refactor a half-built gem.

The Resource interface

Some options for accessing a Post resource:

CoolService::Post.all(page: 1)
CoolService::Post.find(1)
CoolService::Post.find(1).update(params)
CoolService::Post.approve(1)

This quacks like ActiveRecord. That’s a problem. Other developers will assume it behaves like ActiveRecord—lazy loading, dirty tracking, persistence callbacks. It doesn’t. Confusion follows.

CoolService.posts(page: 1)
CoolService.posts.get(1)
CoolService.posts.update(1, params)
CoolService.posts.approve(1)

.posts returning something you call methods on is unclear. What is that object? What’s its lifecycle?

CoolService::Post.list(page: 1)
CoolService::Post.retrieve(1)
CoolService::Post.update(1, params)
CoolService::Post.approve(1)

This is the one. Actions are explicit verbs. Nothing resembles ActiveRecord. It’s grepable. Stripe uses this pattern—and they’ve iterated on client design more than most.

Collection vs. member routes

REST gives you two flavors of custom routes:

Collection routes hit the resource root:

GET /posts/unread

Member routes hit a specific resource:

GET /posts/1/comments
POST /posts/1/approve

For member routes, you have options:

CoolService::Post.retrieve(1).approve(params)

I don’t hate this. The chained call makes the relationship clear.

CoolService::Post.approve(1, params)

Also fine—but it opens a question. If we have Post.approve(1), do we also allow Post.approved for listing approved posts? Consistency matters. Pick a pattern and hold the line.

We’ll go with class methods for now. They’re explicit about what’s happening.

Scaffolding the gem

We’re building against JSONPlaceholder—a fake REST API for testing. The gem gets named Typicode.

$ bundle gem typicode

The gemspec:

Gem::Specification.new do |spec|
  spec.name          = "typicode"
  spec.version       = Typicode::VERSION
  spec.authors       = ["Josh Brody"]
  spec.email         = ["git@josh.mn"]

  spec.summary       = "An example REST client."
  spec.description   = spec.summary
  spec.homepage      = "https://github.com/joshmn/typicode-example"
  spec.license       = "MIT"
  spec.required_ruby_version = Gem::Requirement.new(">= 2.3.0")

  spec.metadata["homepage_uri"] = spec.homepage
  spec.metadata["source_code_uri"] = spec.homepage
  spec.metadata["changelog_uri"] = spec.homepage

  spec.files         = Dir.chdir(File.expand_path('..', __FILE__)) do
    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
  end

  spec.require_paths = ["lib"]
  spec.add_dependency 'http', '~> 4.4'
end

We’re using httprb for HTTP. HTTParty is fine too—there truly is no party like an HTTParty—but httprb’s benchmarks are compelling.

A small convenience patch

HTTParty has #parsed_response. httprb doesn’t. A quick monkey-patch in lib/http_ext.rb:

class HTTP::Response
  def parsed_response
    JSON.parse(to_s)
  end 
end

Then in lib/typicode.rb:

require 'typicode/version'

require 'http'
require 'http_ext'

module Typicode
  class Error < StandardError; end
end

Configuration

Two approaches to config:

Typicode.config.api_key = "my-secret"

Or block-style:

Typicode.configure do |config|
  config.api_key = 'my-secret'
end

Block configs shine when you have many options. We have two: api_key and endpoint. Simple accessors win.

lib/typicode/config.rb:

module Typicode
  class Config
    attr_accessor :api_key
    attr_accessor :endpoint
  end 
end

Wire it up in lib/typicode.rb:

require "typicode/version"

require 'http'
require 'http_ext'

require 'typicode/config'

module Typicode
  class Error < StandardError; end

  def self.config
    @config ||= Config.new
  end
end

And a spec in spec/typicode/config_spec.rb:

require 'spec_helper'

RSpec.describe Typicode::Config do
  context '#endpoint' do
    it 'is nil by default' do
      expect(described_class.new.endpoint).to be_nil
    end

    it 'is writable' do
      expect(described_class.new).to respond_to(:endpoint=)
    end

    it 'is readable' do
      expect(described_class.new).to respond_to(:endpoint)
    end
  end

  context '#api_key' do
    it 'is nil by default' do
      expect(described_class.new.api_key).to be_nil
    end

    it 'is writable' do
      expect(described_class.new).to respond_to(:api_key=)
    end

    it 'is readable' do
      expect(described_class.new).to respond_to(:api_key)
    end
  end
end

Run it:

$ rspec

Typicode::Config
  #endpoint
    is nil by default
    is writable
    is readable
  #api_key
    is nil by default
    is writable
    is readable

8 examples, 0 failures

A playground for smoke testing

Real tests with mocks come later. For now, a quick script to verify things work against the live API.

$ gem install pry

Create playground.rb at the root:

$LOAD_PATH.unshift File.dirname(__FILE__) + '/lib'

require 'pry'
require_relative './lib/typicode.rb'

puts "hello"
$ ruby playground.rb
hello

Good enough.

Building the first resource

JSONPlaceholder gives us standard REST operations. The /posts endpoint returns 100 post objects. Let’s start naive and refine.

lib/typicode/post.rb:

module Typicode
  class Post 
    def self.list
      response = HTTP.get("https://jsonplaceholder.typicode.com/posts")
      response.parsed_response 
    end
  end
end

Require it and test:

puts Typicode::Post.list.size
$ ruby playground.rb
100

Add the remaining CRUD operations:

module Typicode
  class Post 
    def self.list(params = {})
      response = HTTP.get("https://jsonplaceholder.typicode.com/posts", params: params)
      response.parsed_response 
    end
    
    def self.retrieve(id)
      response = HTTP.get("https://jsonplaceholder.typicode.com/posts/#{id}")
      response.parsed_response 
    end
    
    def self.update(id, params)
      response = HTTP.patch("https://jsonplaceholder.typicode.com/posts/#{id}", json: params)
      response.parsed_response 
    end
    
    def self.delete(id)
      response = HTTP.delete("https://jsonplaceholder.typicode.com/posts/#{id}")
      response.parsed_response 
    end
  end
end

This works. It’s also tedious to maintain.

Extracting the HTTP client

Every method repeats the same pattern: build URL, make request, parse response. That’s a code smell.

lib/typicode/client.rb:

module Typicode
  class Client 
    def self.execute_api_request(verb, url, **args)
      HTTP.send(verb, url, **args)
    end 
  end
end

The **args splat lets us pass through any options without knowing what they are—:params, :json, :headers, whatever.

Now post.rb becomes:

module Typicode
  class Post
    def self.execute_api_request(verb, url, **args)
      Typicode::Client.execute_api_request(verb, url, **args)
    end

    def self.list(params = {})
      response = execute_api_request(:get, "https://jsonplaceholder.typicode.com/posts", params: params)
      response.parsed_response
    end

    # ... other methods follow the same pattern
  end
end

Better. But we’re still hardcoding the base URL everywhere.

Update the client to use the configured endpoint:

def self.execute_api_request(verb, path, **args)
  HTTP.send(verb, "#{Typicode.config.endpoint}#{path}", **args)
end

Now methods just pass paths:

def self.list(params = {})
  response = execute_api_request(:get, "/posts", params: params)
  response.parsed_response
end

Configure before use:

Typicode.config.endpoint = "https://jsonplaceholder.typicode.com"
puts Typicode::Post.list.size

If you needed authentication—and you probably do—the client is where it lives:

def self.execute_api_request(verb, path, **args)
  HTTP.auth("Bearer #{Typicode.config.api_key}").send(verb, "#{Typicode.config.endpoint}#{path}", **args)
end

The Resource base class

Multiple resources will share the same operations. Time for inheritance.

lib/typicode/resource.rb:

module Typicode
  class Resource
    def self.execute_api_request(verb, path, **args)
      Typicode::Client.execute_api_request(verb, path, **args)
    end

    def self.list(params = {})
      response = execute_api_request(:get, "/#{collection_path}", params: params)
      response.parsed_response
    end

    def self.retrieve(id)
      response = execute_api_request(:get, "/#{collection_path}/#{id}")
      response.parsed_response
    end

    def self.update(id, params)
      response = execute_api_request(:patch, "/#{collection_path}/#{id}", json: params)
      response.parsed_response
    end

    def self.delete(id)
      response = execute_api_request(:delete, "/#{collection_path}/#{id}")
      response.parsed_response
    end
  end
end

The collection_path method is the key abstraction. Each resource defines its own:

module Typicode
  class Post < Resource 
    def self.collection_path
      "posts"
    end
  end
end

Why not derive the path from the class name? Because pluralization is hard without ActiveSupport, and explicit is clearer anyway. When the API uses /blog_entries instead of /blog-entries, you’ll be glad you can just set it directly.

From hashes to objects

Returning raw hashes works but limits what you can do later. You can’t add methods. You can’t introspect. You can’t define behavior.

Two options for initialization:

  1. Hardcode attributes for each resource
  2. Dynamically define attributes from the response

Hardcoding works if your API never changes. APIs change.

Dynamic initialization in resource.rb:

def initialize(values)
  values.each do |k,v|
    self.class.attr_reader k.to_sym 
    instance_variable_set(:"@#{k}", v)
  end
end

Each key becomes an attr_reader and an instance variable. The object shape matches whatever the API sends.

Update retrieve to return an object:

def self.retrieve(id)
  response = execute_api_request(:get, "/#{collection_path}/#{id}")
  new(response.parsed_response)
end
$ ruby playground.rb
#<Typicode::Post:0x00007fdfff024d60>

It’s an object now.

Ruby-friendly attribute names

JSONPlaceholder uses camelCase: userId, not user_id. This is fine for JavaScript. For Ruby, it’s friction.

Options:

  1. Pray the API never changes and hardcode transformations
  2. Auto-underscore everything
  3. Explicit mapping

Prayer isn’t a strategy. Auto-underscoring breaks when the API has genuinely weird keys like XMLHttpRequest or oAuth2Token. Explicit mapping wins.

A simple DSL:

transform_keys userId: :user_id,
               potatoCake: :potato_cake

Implementation in lib/typicode/transform_keys.rb:

module Typicode
  module TransformKeys
    def transformed_keys
      @transformed_keys ||= {}
    end

    def transform_keys(hash)
      @transformed_keys = hash
    end
  end
end

Updated initializer:

require 'typicode/transform_keys'

module Typicode
  class Resource
    extend Typicode::TransformKeys

    def initialize(values)
      values.each do |k,v|
        sym = k.to_sym
        if self.class.transformed_keys.key?(sym)
          name = self.class.transformed_keys[sym]
        else
          name = sym
        end
        self.class.attr_reader name
        instance_variable_set(:"@#{name}", v)
      end
    end
  end
end

Now in post.rb:

module Typicode
  class Post < Resource
    transform_keys userId: :user_id
    
    def self.collection_path
      "posts"
    end
  end
end

post.user_id works. post.userId doesn’t exist.

Proper response handling

We’ve been assuming success. Real APIs fail.

A response handler needs to:

  1. Return an object for successful hash responses
  2. Return an array of objects for successful array responses
  3. Return error objects for 4xx/5xx responses
  4. Handle total disasters gracefully

In resource.rb:

def self.handle_response(response)
  if response.code.between?(200, 399)
    if response.parsed_response.is_a?(Hash)
      new(response.parsed_response)
    elsif response.parsed_response.is_a?(Array)
      response.parsed_response.map { |object| new(object) }
    else
      raise ArgumentError, "unknown handling of response with type #{response.parsed_response.class}"
    end
  else
    begin
      Typicode::ErrorObject.new(response)
    rescue StandardError => e
      Typicode::ReallyBadError.new(e, response)
    end
  end
end

The error classes in lib/typicode/errors.rb:

module Typicode
  class ErrorObject
    attr_reader :code, :message
    def initialize(response)
      @code = response.code
      @message = response.parsed_response['message']
    end
  end
  
  class ReallyBadError
    attr_reader :exception, :response
    def initialize(exception, response)
      @exception = exception
      @response = response
    end
  end
end

ReallyBadError catches the case where even parsing the error response fails. It happens.

Extracting operations into modules

Not every resource supports every operation. Some are read-only. Some don’t support delete. Inheritance alone can’t express this—you’d need a hierarchy of ReadOnlyResource, WriteableResource, FullResource, and so on.

Modules are cleaner. Extract each operation:

lib/typicode/api_operations/list.rb:

module Typicode
  module APIOperations
    module List
      def list(params = {})
        response = execute_api_request(:get, "/#{collection_path}", params: params)
        handle_response(response)
      end
    end
  end
end

Same for Retrieve, Update, Delete.

Resources compose the operations they support:

module Typicode
  class Post < Resource
    extend Typicode::APIOperations::List
    extend Typicode::APIOperations::Retrieve
    extend Typicode::APIOperations::Update
    extend Typicode::APIOperations::Delete

    transform_keys userId: :user_id
    
    def self.collection_path
      "posts"
    end
  end
end

A read-only resource just omits the write operations:

class AuditLog < Resource
  extend Typicode::APIOperations::List
  extend Typicode::APIOperations::Retrieve
  
  def self.collection_path
    "audit_logs"
  end
end

Sending Ruby-style keys back up

When updating, we don’t want users writing camelCase:

Typicode::Post.update(1, { userId: 123 })  # annoying
Typicode::Post.update(1, { user_id: 123 }) # natural

A utility to reverse the key transformation:

lib/typicode/utils.rb:

module Typicode
  class Utils 
    class << self 
      def normalize_attributes(attributes, klass)
        return attributes if klass.transformed_keys.empty?
        
        inverted_transformed_keys = klass.transformed_keys.invert 
        new_attributes = {}
        attributes.each do |k,v|
          sym = k.to_sym 
          if inverted_transformed_keys.key?(sym)
            new_attributes[inverted_transformed_keys[sym]] = attributes[k]
          else
            new_attributes[sym] = attributes[k]
          end 
        end
        
        new_attributes 
      end
    end
  end
end

Update the Update operation:

module Typicode
  module APIOperations
    module Update
      def update(id, params)
        response = execute_api_request(:patch, "/#{collection_path}/#{id}", json: Utils.normalize_attributes(params, self))
        handle_response(response)
      end
    end
  end
end

Now user_id in Ruby becomes userId over the wire.

Non-RESTful routes

Real APIs have routes that don’t fit CRUD. Approving a post. Archiving a record. Publishing a draft.

You could write these one-off:

class Post 
  def self.approve(id)
    response = execute_api_request(:post, "/#{collection_path}/#{id}/approve")
    handle_response(response)
  end
end

But if you have many, a DSL keeps things clean:

def self.member_method(name, verb:, path: nil, options: {})
  path ||= name.to_s
  
  define_singleton_method(name) do |id, **args|
    response = execute_api_request(verb.to_sym, "/#{collection_path}/#{id}/#{path}", **args)
    handle_response(response, options)
  end
end

Usage:

class Post < Resource
  member_method :approve, verb: :post
  member_method :archive, verb: :post
  member_method :publish, verb: :post, path: 'make_live'  # when the path differs from the method name
end

Calling it: Typicode::Post.approve(1).

Collection-level custom routes work the same way:

def self.collection_method(name, verb:, path: nil)
  path ||= name.to_s
  
  define_singleton_method(name) do |**args|
    response = execute_api_request(verb.to_sym, "/#{collection_path}/#{path}", **args) 
    handle_response(response)
  end
end

Nested resources

Posts have comments. The API exposes them at /posts/1/comments.

First attempt with member_method:

class Post < Resource
  member_method :comments, verb: :get
end
Typicode::Post.comments(1).first
# => #<Typicode::Post:0x00007fd52a81ce80 @postId=1, @name="...", @email="...">

That’s a Post object, not a Comment. The response handler doesn’t know any better—it uses self, which is Post.

Fix: pass an object_class option through handle_response:

def self.handle_response(response, options = {})
  object_class = options.fetch(:object_class, self)
  
  if response.code.between?(200, 399)
    if response.parsed_response.is_a?(Hash)
      object_class.new(response.parsed_response)
    elsif response.parsed_response.is_a?(Array)
      response.parsed_response.map { |object| object_class.new(object) }
    else
      raise ArgumentError, "unknown handling of response with type #{response.parsed_response.class}"
    end
  else
    # error handling unchanged
  end
end

Update member_method to accept and pass options:

def self.member_method(name, verb:, path: nil, options: {})
  path ||= name.to_s

  define_singleton_method(name) do |id, **args|
    response = execute_api_request(verb.to_sym, "/#{collection_path}/#{id}/#{path}", **args)
    handle_response(response, options)
  end
end

Now:

class Post < Resource
  member_method :comments, verb: :get, options: { object_class: Comment }
end
Typicode::Post.comments(1).first
# => #<Typicode::Comment:0x00007f997d80d040 @postId=1, @name="...", @email="...">

Correct type.

What we built

A gem that:

  • Has an explicit, intentional API that doesn’t pretend to be ActiveRecord;
  • Transforms keys between Ruby conventions and API conventions;
  • Returns typed objects instead of hashes;
  • Handles errors with dedicated error types;
  • Composes operations per-resource;
  • Supports custom routes through a declarative DSL;
  • Handles nested resources cleanly

The patterns here scale. Stripe’s Ruby client uses most of them. So do the clients for Twilio, GitHub, and most well-designed API wrappers.

Tradeoffs

This approach requires more upfront work than calling RestClient directly. If you’re prototyping or integrating with an API you’ll call twice, skip the ceremony.

We also haven’t covered: pagination helpers, rate limiting, request retries, webhook signature verification, or test fixtures. Each deserves its own treatment.

The full source is at github.com/joshmn/typicode.

Stay in the loop

Occasional essays on design, tools, and the craft of building things. No spam, unsubscribe anytime.

Ambient weather

The background of this site reflects the current weather and time of day in Saint Paul. The orbs shift in color and behavior based on what's happening outside my window.

Learn more about how this works