Do you really need a custom API to deliver content for your website and fetch it dynamically?

May 15, 2019 - IT

How often do you have to build an API to deliver content which should be displayed on a website? It can be anything like a blog, company page, product page, documentation, everything where the data does not change very frequently.

Even if a website is simple, you have to create and manage a lot of things.

Let's think about a very simple website (JS) where you have a couple of pages, blog, contact form. Your client wants to have a possibility to edit this data and because the budget and time are limited (as always), you decide to use Contentful as a CMS. The website can have a huge number of visitors so it would be nice not make requests to Contentful every time when a user visits a page. What do we need to do?

  1. We have to create an API which can connect to Contentful and get the required data. Let's call it a proxy.

  2. We have to cache the data so we need to set up a Redis and of course implement this feature.

  3. We should also cache the images so we can store them on our side.

  4. We should remove not used images from our storage to reduce the cost.

  5. We need to invalidate the cache if there is any change in Contentful.

  6. It would be nice to have unit tests for our API.

  7. We have to configure CI/CD.

  8. Setup API monitoring.

  9. Create a website and use our API to get content.

  10. A client may want some changes, like content transformations or some special changes in content, based on Contentful data. Every time we have to edit our API and the website.

  11. Everyone can break the website by removing the required data, changing the data structure, etc. It's not easy to prevent situations like this. Someone can remove or edit content by mistake, API can get new data, cache it and the website can have issues with displaying it.

  12. We have to host our API, caching server, images, etc.

  13. API should be scalable and secure.

It's not a rocket sience but it requires some time.

The question is, can we do it better?

Of course we can!

We can use Static Site Generator.

#What is that?

In general, it's a framework which can take your data and build a static website. It's a really popular approach and there is a lot of ready to use generators.

Let's try to plan how can we create the same website using a static site generator.

#Which framework should we use?

That's a good question. In my opinion, it depends on what you need and what programming language you want to use. There is a lot of frameworks which can convert markdown files to create a website but in our case, we want to use Contentful. Not every static site generator has a ready to use support for Contentful.

I don't remember when I started using static site generators but I was using Jekyll, Pelican and Hugo. Recently, I realized that I don't like creating a frontend using things like Python, Jinaj2, Go templates, etc. When I have to create a website I like to use javascript frameworks because I am enjoying it more.

A few days ago I decided to try Gatsby and I really like it! You can check Gatsby features here.

Gatsby is a free and open source framework based on React that helps developers build blazing fast websites and apps

Gatsby is also a JAMStack.

JAMstack: noun \’jam-stak’\ Modern web development architecture based on client-side JavaScript, reusable APIs, and prebuilt Markup.

I do not have much experience writing applications in React so I was a little afraid of it. Fortunately, it turned out it's very simple. You don't need to be a React ninja. If you know javascript, I would say that you need 2 hours to learn the basics of React and you can start work with Gatsby.


#What should we do with Gatsby to create our website?

  1. We can build a frontend using JS (React) and CSS.
  2. We can use a Contentful plugin to get all the data from Contentful (even images).
  3. We can use GraphQL to inject the content to our site.
  4. We can build our site as a static files.
  5. We can deploy it to CDN.

#What's the advantage?

  1. We don't need a custom API, caching, servers (backend), monitoring, etc.
  2. We will make requests to Contentful only during the build time, nothing more.
  3. If the Contentful data is broken, the website will not build so we won't break it for users.
  4. It's a static page so it's secure.
  5. It's fast, it can be served over a CDN.
  6. Better performance.
  7. Development is faster because you have to build only a website.
  8. Hosting is cheaper and you don't need a lot of maintenance.

#How can we host and update the site?

We can use Netlify. I'm using it for my blog and it's really simple and does everything that I need. To host a static site you need only Netlify. They have CI/CD, automatic (free) HTTPS, Netlify Edge (~CDN) to host our site, functions, forms, image resize service and more.

  1. Every time when someone edits data in Contentful, we can run a webhook and inform Netlify about new changes.
  2. Netlify can get our website from a git repository, fetch the fresh data from Contentful, build the website with new data and deploy it.
  3. The same with changes in a git repository. Netlify can rebuild a site on every change.

#Can we use other data sources than Contentful?

Yes, we can. If we have an API where we can get our data, we can use it to generate our website. Gatsby has ready plugins for PostgreSQL, Instagram, custom API, Markdown, Vimeo, Google Photos, Trello, Contentful, Wordpress, Drupal, Medium, Twitter, and many many more.

#Do we have to use Netlify?

No. You can use everything which allows you to serve static files. It can be AWS Amplify, Github Pages, Google Cloud Storage bucket, AWS S3, Firebase,, and even your own Nginx server on a Raspberry Pi.

#Do I need GraphQL server for the data?

No. Let's use Contentful plugin as an example. It knows how to fetch the data from Contentful and using Gatsby it prepares GraphQL data for you. You can also use development server to test queries.

#Can I use dynamic data, make queries, etc?

Yes, why not. It's a javascript. You can have static data and also make queries to get dynamic data from other sources if you need.

#Is it difficult to create a website using Gatsby?

I would say no. Without React experience it took me about 6 hours to create this blog. I refreshed my knowledge about React, created a very simple (by purpose) layout using JSX and Emotion (CSS-in-JS!) for the first time. I learned how to use Gatsby to convert all my blog posts written in Markdown and get the data using GraphQL. Using Gatsby plugins my blog has RSS, sitemap, lazy loading for images, mulitple image sizes for different screen resolutions, favicons, it embeds Twitter, Youtube and other services automatically, etc.

Look at the Lighthouse results!

Lighthouse results
Lighthouse results