In this series of articles I will share my own experience working with microservices on Google Cloud Platform, the different options that we have, pros, cons and limitations of each of the services in GCP.
For this one, we'll be focusing on App Engine, then will move on to troubleshooting microservices (check it here) and microservice security (coming soon).
If you are here it is because you are already deciding about what flavor is better for your use case. So let get this going.
What is App Engine?
AppEngine was the first serverless solution of Google Cloud, its principal objective being to enable developers to focus only on what they truly love: the code.
Google would handle the infrastructure and the load for you, helping to scale up or down your microservice depending on the amount of requests. The obvious here is that App Engine has a smooth integration with all the other services on GCP.
App Engine is made for those companies that don't have a big sysadmin team, and are pretty much made by a bunch of developers who would like to deploy new features faster and easier.
AppEngine has two main flavors for you to build your own apps: standard environment and flexible environment.
App Engine: Standard Environment
This environment should be enough for the majority of the use cases as this is a fully-managed serverless solution.
Google will handle the infrastructure and the elasticity of your application or service, with the standard env having support for the most important programming languages including JAVA, NodeJS, Go, Python, PHP and Ruby for specific versions.
This is actually key when you choose between the standard and flexible, as not all the different versions of the programming languages are supported in the standard environment. There is another way for you to run the latest JDK 14 in App Engine, which is the flexible environment.
Benefits of App Engine Standard Environment:
- There is a free tier that developers will love 🙂
- Application can scale down to 0 if it is not receiving traffic.
- Fast deployment.
Disadvantages:
- You can’t access the infra.
- You have to stick to the versions that are supported by the standard environment.
- First call to the service after a 0 traffic period will take a while as all the infra will be created on the fly.
- Doesn’t support websockets.
So when should you use the App Engine Standard Environment?
- If you want to pay only for what you need and when you use it.
- More suitable for applications that will undergo no-usage times.
- If your team doesn’t have people who feel comfortable managing infrastructure.
App Engine: Flexible Environment
Well all this sounds pretty could isn’t it?
But what if you want to have more control on the infrastructure without handling everything by yourself?
This is where the flexible environment steps in, allowing you to customize the infra, as it relies on Compute Engine (the IaaS in GCP). You can actually add ssh to the infra, troubleshoot directly there, and the scalability is still handled by Google for you.
This also covers the case where you don't want to run the last stable version of a language like NodeJS (the last supported version in the standard environment is the 14). On the flexible environment, you can run whatever version you want.
Benefits of App Engine Flexible Environment:
- Full access to the infrastructure of your applications.
- Ability to customize the runtime.
- Support for any version of your favorite language version.
- Websockets support.
Disadvantages:
- Deployment time will take more time as google will need to bake and deploy the compute engine nodes every time.
- No free tier is available.
- Is not able to scale down to 0 (minimum is one instance).
- Flexible environment virtual machines are restarted on a weekly basis for patching or security upgrades.
When should you use the App Engine Standard Environment?
- When your application requires you to customize the runtime.
- If you need to use a higher or lower version of a programming language that is not supported by the standard environment.
- If your application or service will receive traffic all the time.
- If your application uses websockets.
Practical Example
Let's take this scenario: a sales company requires that each time a new invoice is generated by their ERP system, several teams should receive a notification over email, SMS or chat with relevant information.
Out of these, we’ll only create the email service in NodeJs, for the purpose of the article.
1. Solution Architecture
2. Architecture Description
Distributed systems usually communicate using an asynchronous mechanism and GCP provides support for the publish/subscribe pattern called Cloud PubSub (to be covered in a different article).
In our scenario, every time a new file is created in cloud storage, a new notification will be delivered to the topic, at the same time the topic will push a new notification to the email service and the SMS service both in app engine.
3. Solution Development
Time to unwrap the configuration in GCP (note: you should have gcloud SDK well configured and have some understanding about cloudshell).
Time to unwrap the configuration in GCP (note: you should have gcloud SDK well configured and have some understanding about cloudshell). Here you can find how to configure it by yourself. I highly recommend you use cloudshell for this part.
3.1. Bucket Creation
Bucket name: invoices-docs-testing (remember, this name should be unique globally).
Type: Multiregional
Retention policy: no
gsutil mb gs://invoices-docs-testing
3.2. PubSub Topic Creation
Topic name: invoices-docs-topic
gcloud pubsub topics create invoices-docs-topic
3.3. Configure the Bucket Permissions
Assign the IAM permission pubsub.publisher to the service account for the specific topic (invoices-docs-topic). This way, the cloud storage will be able to push messages to the topic.
Now, the service account from cloud storage is able to push messages to the invoices-docs-topic.
3.4. Configure the notification
gsutil notification create -t invoices-docs-topic -f json gs://invoices-docs-testing
Output:
3.5. Email service in NodeJS
As you can see, there is a simple piece of code that will receive the notification coming from pubsub and will send an email notifying about the new invoice arriving.
As this is for testing purposes, our service will only log the email message. Here is the code reference.
Index.js
const express = require('express');
const app = express();
const bodyParser = require('body-parser');
app.use(bodyParser.json());
const port = process.env.PORT || 8080;
app.listen(port, () => {
console.log('Listening on port', port);
});
app.get('/', async (req, res) => {
console.log(`Email Service: health check`);
res.status(200).send('Health check..');
})
app.post('/', async (req, res) => {
const notification = decodeBase64Json(req.body.message.data);
try {
console.log(`Email Service: Report ${notification.id} trying...`);
sendEmail();
console.log(`Email Service: Report ${notification.id} success :-)`);
res.status(204).send();
}
catch (ex) {
console.log(`Email Service: Report ${notification.id} failure: ${ex}`);
res.status(500).send();
}
})
function decodeBase64Json(data) {
return JSON.parse(Buffer.from(data, 'base64').toString());
}
function sendEmail() {
console.log('Sending email');
}
Package.json
{
"name": "email-service",
"description": "Email Service in Node.js sample for Google App Engine Standard Environment.",
"version": "0.0.2",
"private": true,
"license": "Apache-2.0",
"author": "Pablo Portillo",
"repository": {
"type": "git"
},
"engines": {
"node": ">=14.0.0"
},
"scripts": {
"start": "node index.js",
"test": "mocha --exit test/*.test.js"
},
"dependencies": {
"express": "^4.17.1"
},
"devDependencies": {
"mocha": "^8.1.3",
"supertest": "^6.0.0"
}
}
3.6. Email Service Deployment
In order to deploy a service or application in App Engine, we need to create a YAML file that will help App Engine to know some details about it. The file name shall be app.yaml.
runtime: nodejs14 # or another supported version
service: default
3.7. Execute the following commands
npm install
npm start
Output of the command is:
Select web preview here:
It seems all is working well, so let's deploy our service in the app engine.
gcloud app deploy
gcloud app browse
Copy the target URL and execute this command to verify that all is working as expected.
curl target-url
curl https://adept-fountain-305916.uc.r.appspot.com
You should see this output in the cloud shell:
Health check...
3.8. Create the subscription
gcloud pubsub subscriptions create email-service-sub --topic invoices-docs-topic --push-endpoint=https://adept-fountain-305916.uc.r.appspot.com
3.9. Upload some files to the bucket to see how everything works
Then go the Cloud Logging and select our email service resource, our logs will be there as AppEngine provides support for StackDriver.
After this, you should see the file name and some details corresponding to the PDF that was just uploaded.
Conclusion
App Engine is a very good way to implement microservices in Google Cloud, given all the features provided: autoscaling, scale down to 0, free tier, logging and monitoring with cool tools debugger or profiler that will make your life easier.
If you truly love to code microservices without worrying about the underlying layer (just like me), you will love App Engine.
The only thing that I would say is the biggest downside is that all your services should be part of one application at least within the same project on GCP, so if your goal is to develop microservices for multiple solutions you will need to create one project each, which can be difficult to manage at scale.
Logging, monitoring and security will be covered in the following articles, so stay tuned or subscribe to the newsletter (check below 👇).
