This blog post is intended to be an illustration of my work for GSoC Final Evaluation.

Overview

Vert.x is a toolkit for the Java Virtual Machine enabling the implementation of reactive, highly concurrent, polyglot applications. The idea of the project is to implement a set of applications using the Vert.x toolkit to illustrate how different classes of applications can be done. This project aims to provide guidelines to Vert.x users to implement various applications.

The blueprint project contains three parts: Todo Backend, Vert.x Kue and Online Shopping Microservice. Both runnable code and very detailed documents and tutorials (both in English and Chinese) are provided.

Vert.x Blueprint - Todo Backend

The project repository: sczyh30/vertx-blueprint-todo-backend.

This blueprint is a todo-backend implementation using Vert.x and various persistence (e.g. Redis or MySQL). It is intended to be an introduction to basic Vert.x web RESTful service development. From this blueprint, developers could learn:

• What is Vert.x and its basic design
• What is and how to use Verticle
• How to develop a REST API using Vert.x Web
• How to make use of asynchronous development model
• Future-based reactive asynchronous pattern
• How to use persistence such as Redis and MySQL with the help of Vert.x data access components

The tutorial documents:

• The tutorial document (English)
• The tutorial document (Chinese)

Vert.x Blueprint - Vert.x Kue

The project repository: sczyh30/vertx-blueprint-job-queue.

This blueprint is a priority job queue developed with Vert.x and backed by Redis. It’s a Vert.x implementation version of Automattic/kue that can be used in production.

Feature document of Vert.x Kue is available here: Vert.x Kue Features.

This blueprint is intended to be an introduction to message-based application development using Vert.x. From this blueprint, developers could learn:

• How to make use of Vert.x Event Bus (distributed)
• How to develop message based application with Vert.x
• Event pattern of event bus (Pub/sub, point to point)
• How to design clustered Vert.x applications
• How to design and implement a job queue
• How to use Vert.x Service Proxy
• More complicated practice about Vert.x Redis

The tutorial documents:

• Vert.x Kue Core Tutorial - English Version
• Vert.x Kue Web Tutorial - English Version
• Vert.x Kue Core Tutorial - Chinese Version
• Vert.x Kue Web Tutorial - Chinese Version

Vert.x Blueprint - Online Shopping Microservice

The project repository: sczyh30/vertx-blueprint-microservice.

This blueprint is an online shopping microservice application developed with Vert.x. It is intended to be an illustration on how to develop microservice applications using Vert.x toolkit. From this blueprint, developers could learn:

• Microservice development with Vert.x
• Asynchronous development model
• Reactive and functional pattern
• Event sourcing pattern
• Asynchronous RPC on the clustered event bus
• Various type of services (e.g. HTTP endpoint, message source, event bus service)
• Vert.x Service Discovery
• Vert.x Circuit Breaker
• Microservice with polyglot persistence
• How to implement an API gateway
• Global authentication (OAuth 2 + Keycloak)

And many more things…

The tutorial documents:

• Vert.x Microservice Blueprint Tutorial - Development (English Version)
• Vert.x Microservice Blueprint Tutorial - API Gateway (English Version)
• Vert.x Microservice Blueprint Tutorial - Development (Chinese Version)
• Vert.x Microservice Blueprint Tutorial - API Gateway (Chinese Version)

Final Summary && Future

I really enjoy the work! During the work period I’ve learned a lot about Vert.x, various asynchronous development model, distributed systems, microservice architecture and many more things… This motivates me to investigate new technologies, and the infrastructure of Vert.x. This would be an unforgettable experience.

I’ll keep track of the blueprint project and planning to make some more improvements on the microservice blueprint to illustrate more usage about Vert.x and microservices (e.g. recommendation, Kafka, full-index search). And soon I’d like to contribute to Vert.x Microservice Toolbox to add more features and integration with other technologies. As Vert.x Scala is nearly technical preview, I’m also willing to contribute to Vert.x Scala version.

Thanks

I would like to give my sincere thanks to:

• My mentors, Paulo Lopes and Clement Escoffier. They provided me guidance and gave me a lot of help during the work :-)
• The Vert.x Community.
• The Google Summer of Code team, for this wonderful experience.

Simplified architecture

Some of the unecessary components has been removed:

• Payment Microservice: The logic should be refactored in order to fit for real production logic.

New diagram:

API Gateway

As suggested by Clement and Paulo, I use HTTP-HTTP pattern for API Gateway eventually. The API gateway itself is using HTTPS, but it communicates REST endpoints via HTTP.

So the API Gateway is responsible for:

• Dispatching requests to corresponding endpoint.

How to implement this?
My solution: When publishing endpoints to the service discovery infrasture, every endpoints must provide an api-name to identify the api, then when a request arrives from the gateway, the gateway will get the prefix pattern /api/:name/* and find any corresponding endpoints via discovery, then consume the HTTP endpoint.

• Simple load balancing
• Health check
• Failure handling (using Vert.x Circuit Breaker)
• Global auth state storage(in route context and session scope)

Auth

It’s convenient to use Keycloak via Vert.x OAuth2.

Objectives

In this week, I’m planning to do the following stuff:

• Architecture enhancement
• A simple SPA frontend for the microservice blueprint (use Vue.js)
• Monitor dashboard
• A simple recommendation component
  • Maybe I should write a individual component providing Neo4j integration with Vert.x
• Run the application with Docker Compose
• Update tests
• Update documents and tutorials

Issues

Solved: Global auth

How to manage global auth in a microservice application. For example, we login in from the login page and our login data is saved in the session. When we visit resources that are protected behind the auth, how we validate the user info (e.g. via session data or use SSO)?

Solved: Use OAuth 2 + Keycloak is a good point. The API gateway is responsible for storing authentication principal and validation.

Thought: How to integrate well with SPA frontend?

Solved: Need a better mechanism for API gateway

Solved: use HTTP-HTTP pattern. The API Gateway is a individual verticle and is responsible for dispatching requests, managing global authentication, handling failure, health checking and simple load balancing. The API Gateway itself uses HTTPS, but it interacts with the downstream REST endpoints via HTTP.

Achievement

• Refactored the architecture of the online shopping microservice, simplified some components (more features will be added into next version)
• Enhanced the logic of the entire online shopping process
• Code refinement and bug fixed
• Added some tests (some tests are in local and weren’t committed as it requires some local envs)

The blueprint is an online shopping microservice application. You can treat it as a very simple C2C market like eBay.

Here is a diagram illustrating the microservice architecture:

Currently I designed these components:

• account-microservice: Component for user account
• product-microservice: Component for products we sell and buy
• store-microservice: Component for individual stores in the market
• checkout-microservice: Compoment for order checkout
• payment-microservice: Compoment for payment transaction processing
• order-microservice: Compoment for order dispatching, processing, analyzing and storage
  • NOTE: this component should be divided into several sub-components
• recommendation-microservice: Compoment for simple recommendation for friends and favorite products with Neo4j
• cache-infrastructure: Cache component for services
• api-gateway: API gateway (with load balance)
• shopping-ui: A SPA front-end for the microservice
• monitor-dashboard: Dashboard for performance monitoring and backend metrics data demonstration

Here is the user workflow: User login and add products to chart -> Checkout -> Go to payment page and pay for the products -> Payment transaction success, order submitted -> (Wait for delivery...)

All events are sent on the event bus.

Objectives

In this week, I’m planning to do the following stuff:

• Initial work for Online Shopping Microservice Blueprint
• Prepare for next blueprint application (high performance)

Good luck~ :-)

Issues

API gateway

How to implement an API gateway with load balance.

Checkout model

TODO: The checkout-payment-order model should be changed.

Deployment

In our design, the online shopping microservice blueprint contains many components, so we need to pay attention to how to deploy such a microservice application (e.g. use Docker Compose, Kubernetes; Deploy on Openshift)

Order dispatching design

In our current design, orders are sent via clustered event bus. But a real production environment, orders are supposed to be dispatched to message queues with recovery and persistence ability(e.g. Kafka). And the order processing component and order persistence component should pull orders from the MQ.

Integration with other technologies

e.g. Kafka

Achievement

• Initial work for Online Shopping Microservice Blueprint

Objectives

In this week, I’m planning to do the following stuff:

• Finish Chinese documentation of Vert.x Kue
• Work on microservice blueprint
• Learn more about Vert.x Microservice

Notice: Because my semester final exams is pending this week and next week, I’ll have to be busy with them. So I might spend less time on it (have time in the evening). After the final exams I’ll speed up and make a wonderful blueprint!

Good luck~ :-)

The final exams finally finished!!!

Objectives

In this week, I’m planning to do the following stuff:

• Some extra things to implement or fix:
  • Job active TTL check
  • Job search functionality
  • Render issue with _filter.jade
• Finish documents and tutorials of Vert.x Kue
• Finish tests
• Add Docker Compose file
• Prepare for the microservice blueprint

Good luck~ :-)

Issues

TODO: Job search functionality

This need a full-index engine.

TODO: Run Vert.x Kue in Docker Cluster

Maybe we need some configurations for Hazelcast to support it on Docker.

Achievement

• Finish English documents and tutorials of Vert.x Kue
• Finish part of Chinese documents and tutorials of Vert.x Kue
• Initial design for bluepint microservice

These days my progress is slow…Because I have to prepare for the final exam(7.1 - 7.15) these days…

In this blog post, I’ll give a brief introduction to illustrate how Vert.x Kue works.

In the workflow, first we deploy the KueVerticle in clustered mode and register the JobService on the event bus. The JobService consists of various logic of Job. After that, we could then deploy KueHttpVerticle to provide REST API. Most of the apis calls Future-based methods in Kue. Finally we could deploy our job processing verticle. The verticle usually consists of three parts:

• create Kue instance

When we create Kue instance, the inner JobService proxy and redis client will be created. Also, the static Vertx field in Job class will also be initilized.

• create and save Job

Here we could set job properties and subscribe events from the job address(via event bus).

• process the Job

When we call the process or processBlocking method, Vert.x Kue will create and then deploy some KueWorkers, which is a kind of verticle(or worker verticle, if calling processBlocking).

In KueWorker, we first get Job from Redis backend (zpop operation):

1
2
3
4
5
6
7
8
9
10
11
12
13
14

private void prepareAndStart() {
  this.getJobFromBackend().setHandler(jr -> {
    if (jr.succeeded()) {
      if (jr.result().isPresent()) {
        this.job = jr.result().get();
        process();
      } else {
        throw new IllegalStateException("job not exist");
      }
    } else {
      jr.cause().printStackTrace();
    }
  });
}

If the KueWorker successfully gets the job, it will do process procedure. First we set started_at flag on the job and then save it to the backend. Then we emit the job start event on the event bus(both job address and worker address). Next, we will process the job using the given handler. Simultaneously, we consume done and done_fail events on the job-specific address. If done event received, the job should be completed and we emit complete event. If done_fail event received, the job should be failed so we first do a failed attempt. If attempt failed, then send error event on worker address. Else send failed or failed_attempt event depending on the attempt status.

There are three kinds of address:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

/**
   * Handler address with certain job on event bus
   * Format: vertx.kue.handler.job.{handlerType}.{addressId}.{jobType}
   *
   * @return corresponding address
   */
  public static String getCertainJobAddress(String handlerType, Job job) {
    return "vertx.kue.handler.job." + handlerType + "." + job.getAddress_id() + "." + job.getType();
  }

/**
 * Global worker address with specific event type
 */
  public static String workerAddress(String eventType) {
    return "vertx.kue.handler.workers." + eventType;
  }

  /**
   * Worker address with specific event type and job-uuid on event bus
   * Format: vertx.kue.handler.workers.{eventType}
   *
   * @return corresponding address
   */
  public static String workerAddress(String eventType, Job job) {
    return "vertx.kue.handler.workers." + eventType + "." + job.getAddress_id();
  }

Diagram

The GSoC Midterm Evaluation is coming~ Boost~O(∩_∩)O

Objectives

In this week, I’m planning to do the following stuff:

• Enhance Event system and KueWorker in Kue
  • Implement all basic events in original Kue
  • If possible, do some performance optimization
• Add test cases and fix bugs
• Add more examples
• Finish all Vert.x Kue REST apis
• Adapt Vert.x Kue REST API with original Kue’s UI
• Code refinement
• Initial document and design diagram update
• Learn more about new features in Vert.x 3.3.0

Achievement

• initial-work implementation of Vert.x Kue (including Vert.x Kue REST API and UI)
  • See: sczyh30/vertx-blueprint-job-queue
• Bugs fixed
• Add more examples
• Some refactor and code refinement
• Feature document draft

Objectives

In this week, I’m planning to do the following stuff:

• Core component of Vert.x Kue implementation complete
  • Event system in Kue
  • KueWorker
  • Job
• Basic implementation of Vert.x Kue HTTP component
  • REST API implementation
  • A simple UI (Vert.x Kue UI)
• Refine the code of Vert.x Kue
  • The Job class is messy and need refinement
  • More concise
• Learn more about microservice (as well as Vert.x Microservice Toolbox)
• Write a brief introduction to new features of Vert.x 3.3.0 (in Chinese, will publish in Vert.x China Group)

PS: The Dragon Boat Festival holiday will be coming from Thursday to Saturday this week so I may not spend very much time on work during the holiday~ Have a great week :-)

Issues

Solved: Regex match in route path

In original Kue’s API, there is a route with path /jobs/:from..:to/:order?. I tried this in Vert.x Web(/jobs/:from..:to) but Vert.x Web cannot match it. So I used regex match \/jobs\/([0-9]*)\.\.([0-9]*)(\/[^\/]+)?:

1
2
3
4
5

public static final String KUE_API_JOB_RANGE = "\\/jobs\\/([0-9]*)\\.\\.([0-9]*)(\\/[^\\/]+)?";

// ...

router.getWithRegex(KUE_API_JOB_RANGE).handler(this::apiJobRange);

But nothing matched, which made me confused. And path like /jobs/:from/to/:to can be matched.

Solution: First use a supported pattern to catch string like 1..10 then use regex parse.

Achievement

Implementation of Vert.x Kue core

• Job processing interface KueService now provides two methods: process method do asynchronous procedure(use common verticles), while processBlocking method could do blocking procedure(use worker verticles)
• Add attempt support (could attempt when fails)
  • TODO: need enhancement and test
• Add worker and job cardinality metrics support
• Add support for getting jobs with range(rank)
• Basic refinement of Job class
• Refactored most of async methods to Future based style (in order to support monadic transform and say goodbye to callback hell)
• Fix issue: Sometimes throw java.lang.ClassCastException caused by wrong time flow when processing a job
• Fix issue: Generated json converter sometimes could not distinguish some properties(e.g. id and jobMetrics) correctly

See here: sczyh30/vertx-blueprint-job-queue

Basic implementation of Vert.x Kue HTTP component

• Basic implementation of Vert.x Kue REST API(get job, create job, delete job, get job by range(or state), etc)
• Basic adaption between Vert.x Web and original Kue UI (rendered with vertx-web-templ-jade; still have some problems to solve)

Wrote a brief introduction to new features of Vert.x 3.3.0 in Chinese