Apr 15, 2018
4 min read

AWS Lambda + Go - Showing Medium Posts on My Personal Site

What?

Showing my Medium article list on a static website using an AWS Lambda function built with Golang and the AWS API Gateway.

Why?

I was building a virtual homepage using GitHub pages and wanted to show the list of articles I’ve written on medium. There is no method for this in the official Medium API, but this url https://medium.com/@username/latest?format=json does return a list of articles for a user in JSON format.

This provides a conventient way to retrieve the data, but CORS prevents it from being loaded in the frontend app within the browser. I could proxy this information through a server but I did not want to create and manage one for this, which is why I was using GitHub pages in the first place.

For this, AWS Lambda comes to the rescue! With AWS Lambda you can write a function in any of their supported languages. You can then upload it to the Lambda service either through CLI or the AWS browser console. There is no need to provision or manage a server. It’s also very cheap.

How?

First we will write the function in a file locally, then we’ll create a new Lambda function and upload our local file to it. To access the Lambda function, we’ll create an AWS API Gateway APT to call from our frontend. The full file I used is below, followed by an explanation.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io/ioutil"
	"net/http"
	"strings"

	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-lambda-go/lambda"
)

type Response struct {
	Payload struct {
		References struct {
			Post map[string]Postdata
		} `json:"references"`
	} `json:"payload"`
}

type Postdata struct {
	ID               string `json:"id"`
	Title            string `json:"title"`
	FirstPublishedAt int64  `json:"firstPublishedAt"`
	UniqueSlug       string `json:"uniqueSlug"`
}

func handler(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	fmt.Printf("Processing request data for request %s.\n", request.RequestContext.RequestID)

	resp, err := http.Get("https://medium.com/@searching.nehal/latest?format=json")
	if err != nil {
		fmt.Println(err)
	}

	defer resp.Body.Close()

	body, err2 := ioutil.ReadAll(resp.Body)
	if err2 != nil {
		fmt.Println(err2)
	}
	data := string(body)
	convertedData := strings.Replace(data, "])}while(1);</x>", "", 1)

	var response Response

	json.Unmarshal([]byte(convertedData), &response)

	result, _ := json.Marshal(response.Payload.References.Post)

	return events.APIGatewayProxyResponse{Body: string(result), StatusCode: 200}, nil
}

func main() {
	lambda.Start(handler)
}

We create a file called main.go and import github.com/aws-lambda-go/events and github.com/aws-lambda-go/lambda. We define two structs, Response and Postdata, to parse data from the JSON response of our API call.

Next we define a function handler which takes two parameters: context (runtime information) and request (information about the API call from the AWS API Gateway). We return a response to send back to the API Gateway caller and an error.

Within the function we make a GET request using thenet/http library to the Medium URL (line 33) and convert it to a string (line 40). Then we remove some unwanted text using from the response by using ])}while(1); (Medium adds these to avoid JSON hijacking) at line 45. Then we decode the JSON (line 49) and again encode only the part we need back to JSON (line 51). Finally, we return the response.

The main function is the entry point, and we call the Start function from the lambda package to run our handler function.

Next, we’ll build an executable binary for a linux machine GOOS=linux go build -o main main.go and zip the file zip main.zip main.

Now, let’s create the Lambda function from the AWS browser console. Once logged in, navigate to the Lambda section.

Click the “Create function” button.

Select “Author from scratch” and enter a name for your function, selecting “Go 1.x” as your runtime. Create a new role if you don’t have an existing one, and give the role a name. Give the role “Simple Microservice permissions” in “Policy templates” and then click “Create Function.”

Your function will be created and moved to the “Configuration” page. In the “Configuration” page go to the “Function code” section and upload the zip file. Also provide the name main in the handler input box.

In the “Basic Settings” section on the same page provide a timeout for your function execution — one minute seems to be enough for this function. Lastly, save it.

Now let’s create an API to call the Lambda function. Go to the “API Gateway” section and click “Create API.”

Give the API a name, then click “Create API.”

In the next page select “Create Resource” from the “Actions” dropdown.

Give the resource a name, check the “Enable API Gateway CORS” checkbox, and create the resource.

Now create a GET request from the “Actions” dropdown.

On the next page select “Lambda Function” as the integration type, and choose the Lambda region of your Lambda function. Provide the already created Lambda function name, then click “Save.”

Select OK in the permission modal.

Now select “Deploy API” from the “Actions” dropdown and create a new deployment stage. Give it a name and click “Deploy.”

Navigate to the “Stages” menu and check the GET method, where you will see an “Invoke URL.” This is the URL we will use to call the Lambda function. Let’s try calling that in the browser. We should see a JSON response containing the list of posts from Medium.

From my frontend app I can call this url, parse the response to JSON, and display the posts as a list. You can check out that code here.

You can also apply authentication to this API in various ways (API Gateway Lambda Authorizers, Amazon Cognito User Pools, etc.), and you can test your Lambda function locally via AWS SAM.

I hope this article was helpful. Please provide feedback or comments if you found any errors. Subscribe to my newsletter to get notifications for new posts.

Comments:
If you found any error or have any suggestion let me know in the comments.