Integrate API Gateway with Kinesis Firehose using CloudFormation
24 August 2017
Integrating API Gateway with other AWS Services can be pretty important to increase the scope of an API into other services.
What I wanted to achieve was a cheaper upload mechanism for S3. The easiest way to allow upload through API gateway is to call a Lambda for every API call and then upload the payload into an S3 bucket. But this is rather costly if you increase the throughput from a few single call to a few hundred calls a second.
So what I came up with, was to directly invoke the Kinesis Firehose from an API Gateway. That way I could avoid the cost of the Lambda and the cost of a S3 request per API call.
So here an overview picture of what I am about to build.
The easiest part in CloudFormation is the S3 bucket. This one does not require any specific configuration for this to work.
Now I configured a Firehose.
As a prerequisite I needed a role which would allow the Firehose to actually write into the S3 bucket.
Now the Firehose configuration.
The API Gateway only consists of one method (without any resources), to keep the sample as simple as possible.
Here the API Gateway definition.
Now the tricky part for all of this lies in the method definition (specifically the input mapping).
Now we still need the Gateway role to be defined.
I added a Deployment, so the API would automatically get deployed after it is created.