Blog

  Shawn Smith

Go at Gengo

6 Comments
alignnone

At Gengo, the engineering team is constantly working to create a better experience for our customers and translators. One of the ways we’re doing this is by using more of our favorite new technologies such as Go, an open source programming language. We recently built several services in Go that have improved our speed, reliability and ease of use, including two projects we call GoShip and the Email Consumer.

GoShip

GoShip, the first service that we wrote in Go, is an open source web application that makes it easy to deploy code to servers.

It takes a YAML configuration file containing hostnames, environments (pre-production, production, etc.) and the location of the git repository on remote servers. For each project, GoShip SSHes into its hosts and checks whether the latest git revision differs from the most recent revision on GitHub. If the revisions differ, a deploy button is shown on the page, which runs the deployment command specified for that project. The output is displayed and then stored along with the time and status of the deployment.

For added fun, our Art Director, Tina, even drew an informal mascot for GoShip featuring the Go gopher at the helm of his own pirate ship (pictured above).

The Email Consumer

We also created a service to send emails that uses Iron.io‘s queue service, IronMQ. We call this the Email Consumer, which pulls JSON payloads off of a queue before rendering and sending the email that matches the ID in the payload.

{
    "data": {
        "display_name": "Shawn Smith"
    },
    "email_to": [
        "shawn@example.com"
    ],
    "from_email": "consumer@gengo.com",
    "from_name": "Consumer Test",
    "id": "welcome-c",
    "lc": "ja"
}

In the above example, a new customer signed up on our website, so our web application puts a JSON payload with the id “welcome-c” onto the queue. The Email Consumer consumes the payload and looks up the email subject for the email with the given ID, also rendering the template with the given data. In this case, the data is simply the user’s name. It also localizes the strings in the template based on the language code provided (in this case, it’s Japanese) and sends the email via SendGrid, welcoming the customer to Gengo.

Moving all of our emails into one place helps us easily make consistent style and copy edits. We send over 50 different emails to customers and translators through the Email Consumer, and to date it has sent over 500,000 emails without a problem.

On the backend

We’re moving several backend services from PHP and Python to Go as well. This includes loading and validating jobs in our translator workbench, as well as a port of our glossary service that reduced the total lines of code from 33,000 lines of Python and PHP to 1,700 lines of Go, improving the speed by a factor of 10. We don’t attribute the line count reduction entirely to Go—we removed a lot of legacy code, too—but Go made refactoring the service much simpler.

Our overall experience with Go in production has been very positive, and in the future we plan to move even more of our services to Go. If you’re interested in trying it out, take a look at how to get started and how to write Go code.

Shawn Smith
THE AUTHOR
Shawn Smith
Shawn is a software developer from Boston, Massachusetts. After graduating from Northeastern University, he moved to San Francisco to work for Rackspace, and now works at Gengo in Tokyo. His favorite programming language is Go.
  • Richard Warmington

    Hi. I’m a Japanese-English & Chinese-English translator at Gengo. This is very interesting – I’d never heard of Go before. Actually, I haven’t done any programming for a few years, but now I have a project in mind (in computational linguistics – aligning parallel texts) and I’ve been wondering what language to use. I was using Visual Basic previously, and I’ve been thinking I should learn Python, but Go seems worth checking out. Would it be good for my purposes?

    • Shawn Smith

      Hi Richard,

      Thanks for your comment.

      While Go is really great, I think for your project, Python may be the better choice.

      Python has a plethora of modules for this sort of task; one library that comes to mind for linguistics is NLTK (Natural Language Toolkit): http://www.nltk.org/

      My advice would be to start the project in Python, and if you find that you are not using third party modules as heavily as you anticipated, you can consider rewriting it in Go.

      • Richard Warmington

        Thanks for those tips, Shawn. I’ll have a look at NLTK, and in any case, I think I’ll use Python because I have some familiarity with it already, having looked at it once, a while back.

        • Roman Ganchenko

          Is the GO ready for production? I mean how is ecosystem doing? Are there web frameworks like Rails or Django(I’ve heard of Gorilla) Can long-term projects be started with Go?

          • Shawn Smith

            Hi Roman,

            In our experience, Go has been very stable in our production environments.

            I would also say the ecosystem is growing rapidly. I personally have a bit of experience using the Revel web framework: http://revel.github.io/ in order to get a similar experience to things like Django.

            As for long term projects, Go has been stable for quite a while now and we have found that once we’ve deployed services in Go, we very rarely need to change anything: they just work!

          • Roman Ganchenko

            Awesome! Thank you for sharing your opinion. I was thinking of starting a rails server again(because I’m experienced) but I want to start with Go because iron.io has very solid experience and good feedback(30 servers vs 2 with Go) and TJ Holowaychuk left node.js space to Golang