This is the second in a series of seven posts on service-oriented architecture derived from a workshop conducted by Cloves Carneiro and Tim Schmelmer at Abril Pro Ruby. The series is called SOA from Day One – A love story in 7 parts.
In the previous article in this series, we laid out some of what we understand about Service-Oriented Architecture, and why we think starting with SOA from day one is a good idea.
To bootstrap things, and to get us focussed on some key areas of developing in a SOA, we have gone ahead and built out a set of apps at varying stages of completion. This, and the following 5 posts will focus on showing some of this code. Exercises to add features, or to refactor will be interspersed.
In this part, we introduce the applications that will constitute our tiny SOA. We will show how to set these applications up locally, how to deploy them to heroku, as well as explain a viable development workflow for building applications in a service-oriented set-up.
The functionality of the entire system is to show deals (aka. ‘inventory items’) available in or near a given city. These items can also be organized by a ‘category’ (aka tag), to cater to the various customers’ areas of interest.
Let’s start out by introducing the sample applications, all of which can be found on my github account.
The Consumer-Facing Application
This application is a classic (albeit entirely DB-less) Rails 4 web app, based on Twitter’s bootstrap. The main features it will be exposing are:
- a list of deals (aka. inventory items) in a market (city)
- links to the pages for nearby cities
- a list of inventory items nearby (i.e., in cities within a given range in miles)
- details about a single inventory item
- shows an item’s image, price, and description
- also shows a section with deals nearby
- a page for deals of a particular category (i.e., inventory items that are tagged as belonging to a particular category)
The above Deals application retrieves all its data from the following backend applications.
This is the application’s core service, and it vends, amongst other things
- general inventory information (price, market, title, image url, description)
- a list of all inventory items
- all items anchored in a particular city
- all items in cities nearby a given city
- plus, the various endpoints to add or manipulate inventory items
This service application is responsible for storing tags, and for associating them with inventory items.
It has the following endpoints for tag management:
- show a list of all tags
- show information about a single tag (which includes list of items tagged)
- create / manipulate tags
As a second set of functionality, it exposes endpoints for managing tagged inventory items, namely:
- to return a list of all inventory items tagged with a given tag
- to return a single tag + item combination
- to create an “item is tagged” relationship (i.e., tag an item)
- to delete an “item is tagged” relationship (i.e., ‘untag’ an item)
Finally, this service’s responsibility is to own all city (aka. market) related data (like its name, country, state, lat/long, and cities nearby). It offers endpoints to:
- find a city by ID and show its full information,
- list all cities
- list all cities in a given country
- list all cities near a given city (in a given range)
- create / manipulate cities
Setting up the apps locally
We know, we just told you in part I that we don’t want to ever have developers have to set up all the services on their local machines. This general rule came with a footnote, though: if you would like to make changes to service apps, you will still need to have them set up locally … which is exactly what we are going to be doing from now on.
Clone the repositories for all four applications listed above.
In each of the 3
..._serviceapps, execute the following:
1 2 3 4 5
- In the front-end
As these applications depend upon each other (as described in the previous section), there needs to be configuration to declare these dependencies in each of the apps.
deals application, the only dependency is inventory-service, as it is (quite inelegantly) hardcoded in the application’s ruby code
cities-service will initially have no external service dependencies, but will add reliance on
tags-service at a later stage in its lifecycle.
tags-service is entirely without external service dependencies, and will remain so for the rest of this series of blog post.
As you can see in this example of
inventory-service’s configuration for accessing
tags-service, the application reference heroku instances (owned by me) in the
development environment by default:
1 2 3 4 5
The clear advantage for developers who are just concerned with making changes to
inventory-service is that they will not need to run any of the other dependent services (in this case
tags-service) locally on their development machine: all communication happens into the heroku ‘instances in the cloud’.
However, if you ever need to make changes in several services that depend upon each other (e.g.,
city-service), then you can start both services locally and simply change the service configuration in
city-service to point at a locally running instance of
tags-service (which in this example running on port 3004), like so:
1 2 3 4 5
Using Heroku as your development environment
While there are many ways to get started on a cloud-hosted infrastructure, the rest of this series will use heroku’s infrastructure to show the basics of how a development workflow in a service-oriented environment can work out.
If you are following along and want to try some of the examples in this series of posts by yourself in a cloud-hosted environment, you can get started by following these steps:
- Unless you already have an account with heroku, visit Heroku and sign up for a free developer account
- Next, we are loosely following the “Getting Started with Rails 4.x on Heroku” guide:
- in each of the local git repos for
1 2 3 4 5 6
cities-service DB seed exceed the limit for the “Hobby Dev” pg instance; either trim down the
seeds.rb file to < 10k entries, or upgrade to “Hobby Basic” (~ $9 / month)
Now visit your app via a browser hitting
http://$MY_UNIQUE_HEROKU_NAME.herokuapp.com, or via
heroku open. You should see a swagger UI index page of your service
Here is a list of miscellaneous, useful commands that we have found to come in handy when working with
heroku logs: shows your app’s rails logs
heroku run rails console: bring up a rails console on your application server
heroku maintenance:[on|off]: dis-/enables your app and shows a maintenance page
heroku ps: lists all your app’s dynos, and what they are running
heroku ps:scale web=[0|1]: dis-/enables your web worker dynos
heroku config:set MY_ENV_VAR=foo --remote development: sets an environment variable (e.g., RAILS_ENV) on your app’s server
- We set the
RACK_ENVvariables on some of the heruko apps to
developmentthis way … more later.
- We set the
So, … where are the tests for this code?
Well, there are no tests. Yeah, yeah, we know … we will go into TDD hell, and all.
But seriously, why are there no tests? Here are the main reasons:
- These projects will never see any production traffic.
- The underlying service framework code is well-tested in our production projects; much of what you see in the service repositories is almost exactly the same code as what we are using in our day-to-day jobs
- we are lazy, and we want you to do all of you work for us in a later part of this series where you will be adding some test coverage.
Development and Deployment Workflow
Manage several environments
To effectively develop, test and deploy in a SOA, you will need (at least) three environment ‘fabrics’
a local development machine (running tests and used for developing)
- all your local development is done here
- only check out the applications and services you actually are planning to change
- configuration files (e.g., yml files in your Rails app’s
configdirectory) for all services your application depends upon pointing into the
a remotely hosted
development(or staging) fabric
- remote here means that these services do not run on your local development machine
- all services your local machine depends upon run here, at a well-known domain names; in our example set of applications, the three services run at
- once development is completed locally, new (feature) branches get promoted to here
- data held in data stores for applications deployed here is ‘production like’, so that production conditions can be simulated; it is most likely manufactured seed data, sometimes generated from production data dumps.
- after some quality assurance, code will be promoted to the next (
and finally, the
- this is your stable production level code
- this is running on server instances that the end customers will connect to and exercise
- the data here is the actual production data all your live applications produce and depend upon
How is this implemented?
Looking at the sample applications we have coded up for this series, every app has (yml) configuration files that declare where its dependent services are located:
- for the
productionenvironments (e.g., the respective sections in
developmentsections point at the well-known development fabric instances of the dependent service, while
productionsections point at production services
testsections most likely point at the well-known development fabric instances, or the actual production services
- this way, tests can retrieve (test) data from such dependencies without having to run them all locally
- the service responses can be stored in canned responses for future runs (e.g. using gems like
vcr… about which we will talk later)
Working with Heroku
First off: there is no particular reason why we chose heroku over similar public cloud offerings such as straight AWS EC2 instances, or DigitalOcean, or even private clouds like Cloud Foundry Community.
We like heroku for the reason that it offers comprehensible pricing, as well as good documentation for working with multiple environments.
Here are a list of steps you can follow to set up your own development and deployment workflow when using heroku:
Create your development fabric instance of a service via the
herokucommand line tool
heroku create --remote development
…or rename your current
git remote rename heroku development
Create your production fabric instance of a service
heroku create --remote production
herokuapps run in
productionenvironment mode; to have your
developmentfabric instances all easily point at each other, change their
RAILS_ENVenvironment settings to
development, like so:
heroku config:set RACK_ENV=development RAILS_ENV=development --remote development --app <my_dev_heroku_appname>
herokuignores all branches that are not
master, you need to push you local feature branches to the remote master branches
- I.e., to push a branch called
developmentfabric instance, you need to do:
git push development feature:master
- I.e., to push a branch called
As an example of my current
1 2 3 4 5 6 7
Exercise “Deploying Code”
If you followed along, and want to play around with deploying the sample apps in a distributed multi-fabric environment, we suggest you try tackling the following exercises based on the sample code:
herokufor all three sample services, and adapt your
tags_service, its services’ configuration yml files (
config/cities_service.yml) will need to be pointed at your
developmentfabric instances respectively. Keep your
testyml file entries pointed at your
- Make similar changes to the
cities_servicerepository, so that it will point at the dependent
tags_servicein its respective
- push these changes (e.g., as new git branches) first to your
developmentfabric boxes, and then – after some initial testing – through to the