GDPR compliant forget-me app with Spring Integration (Part 1): Requirements

In preparation for the enforcement of GDPR which becomes final on the 25th of May 2018, I’m creating a simple, open source forget-me app for springuni.com in compliance with Art. 17 GDPR (‘right to be forgotten’). In the first phrase I’d like to support a simple forget-me process driven by the Scatter-Gather messaging pattern using Spring Integration and RabbitMQ, although I’ve got a tons of ideas about making a generally usable, granular and flexible consent management app for fellow bloggers. Before going into coding the app instantly, in this article I just setting the initial requirements for myself and for starting a discussion if someone else is interested in having their own forget-me app.

Non-technical requirements

  1. GDPR is tricky in that sense, that you have to delete folks’ personal data from all your systems, but you still have to be able to prove that you did comply with their request. Nowadays email address is the primary handle used to identify individuals and 90% of my subscribers use Gmail, 5% use Yahoo Mail and the rest use other providers, perhaps they are self hosted. Having that said, the first requirement is to use social login backed by Google’s, Yahoo’s and Facebook’s OAuth2-secured authentication flow. The last one would probably allow those people to login in, who use a customer email address of some kind. Spring Security 5’s new OAuth2 support will be just fine for the purpose.
  2. The app cannot persist personal data, unless it’s absolutely necessary to carry the forget-me process out. Yet, it has to be able to state that the request of a certain individual has been successfully completed. By hashing (probably a SHA-2 variant will do) email addresses and creating a user profile without any personal details this requirement can be satisfied.
  3. Let’s think about the life-cycle of a subscriber’s subscription. Someone signs up for my newsletter, they may unsubscribe and eventually they may ask me to forget all their personal data stored. Given these three uses cases, a subscriber can be in the following states.
    • SUBSCRIBED – denotes an active subscriber, who gave their consent to receive newsletters.
    • UNSUBSCRIBED – denotes an inactive subscriber, who withdrew their consent to receive newsletters.
    • FORGET_PENDING – means that their requested their personal data to be erased with regards to a subscription
    • FORGET_COMPLETED – means that personal data of a subscription has been deleted
    • FORGET_FAILED – means that personal data couldn’t be deleted due to a technical issue

  1. When someone signs up to my newsletter, the forget-me app should be notified and a subscriber and a subscription with status SUBSCRIBED needs to be created there, the same applies when for unsubscribes.
  2. Upon request all systems handling personal data needs to be asked to remove them.
  3. When someone would try to login without being having a subscription, that request can be rejected.

UI design

Eventually these requirements boil down to a simple UI with two pages. As a back-end guy, I just prefer to do the mock-ups quickly without any fuss on a piece of paper. Yes, I know, there are tons of wireframing tools, but nothing as fast as drawing by hand with a pencil.

After logging in,  subscribers can see their current the statuses of their subscriptions and a change log.

Provided that the currently logged in user is in a(n) (UN)SUBSCRIBED state, the Forget Me button becomes active and after a confirmation the process can be started.

Technical requirements

Even before getting into the technical requirements, I’d like to make that mental note to myself, that I’m not allowed to spend more that 8 hours to complete a deployable prototype based on Spring Boot 2 to Heroku.

Software stack

  1. I don’t expect heavy traffic, so it’s perfectly okay and it’s rather desirable that the app go into sleep after being inactive for a while. Heroku makes a Dyno sleep after 30 mins of inactivity.
  2. I’ve got only a single requirement for the database: keep running costs low. I chose PostgreSQL for the prototype as Heroku provides that out of the box and other major cloud providers (eg. Amazon RDS for PostgreSQL and Google Cloud SQL for PostgreSQL) support it as well.
  3. Putting a UI together is always challenging, as there are so many JS frameworks out there. To keep things simple, this is going to be a standard Spring MVC app with Thymeleaf as the template engine and VueJS to handle UI updates.
  4. The app will be connected to message broker receiving subscriber update messages and emitting erasure requests. Heroku supports various RabbitMQ providers, I used CloudAMQP before, probably will stick to that.
  5. Spring Integration will be used to define and execute message flows
  6. In the first phrase only a single system MailerLite will be supported, but I design a plugin-able architecture to be able to add other providers easily

Message flows

There will be two distinct messages flows, one for receiving subscriber updates from other systems from through a webhook and the other one which handles deletion requests.

In the first case an external system submits a piece of JSON describing a SUBSCRIBE or an UNSUBSCRIBE event and upon receiving that, the forget-me app registers a new subscriber and a subscription specific to that data handler. In case of MailerLite, the payload looks like this. Messages transformers will be used to translate the provider specific incoming data and extract the email address and the event’s type.

Upon requesting data erasure, the subscriber’s request will be put into a message queue and that request will be broadcast to all the registered data handlers. Thereafter they will reply back with either an ACK or a NACK and in both cases the result will be recorded.

Messages exchanged

Note: highlighted fields are messages headers, they’re just displayed together with the message body for brevity.

Webhook

As I mentioned above, data provided through the webhook varies widely, but with provider specific message transformers they’re going to be transformed to the following generic message.

{
"event_id":"294cab6a-d915-4654-9fc9-afc6410bf62c",
"event_time":1526892561,
"event_type":"webhook",
"data_handler_name": "mailerlite",
"data_handler_id":"5d4ae976-e397-4e62-9458-2a0f51c8c988", 
"subscriber_email":"test@springuni.com",
"subscriber_status":"SUBSCRIBED"
}

Forget request

{
"event_id":"294cab6a-d915-4654-9fc9-afc6410bf62c",
"event_time":1526892561,
"event_type":"forget-request",
"data_handler_name": "mailerlite",
"subscription_id":"99d2a1a3-84d6-48fd-8ef9-e4681f743958", 
"subscriber_email":"test@springuni.com"
}

Forget response

{
"event_id":"294cab6a-d915-4654-9fc9-afc6410bf62c",
"event_time":1526892561,
"event_type":"forget-response",
"data_handler_name": "mailerlite",
"subscription_id":"99d2a1a3-84d6-48fd-8ef9-e4681f743958", 
"acknowledged": true
}

REST API

The app is going to have a REST API with which the UI will communicate and also it will be used for performing administrative tasks as well.

MethodEndpointAccess level
GET/api/subscribersadmin
POST/api/subscribersadmin
GET/api/subscribers/{subscriber_id}admin
POST/api/subscribers/{subscriber_id}/forgetadmin
GET/api/subscribers/meuser
POST/api/subscribers/me/forgetuser
GET/api/datahandlersadmin
POST/api/datahandlersadmin
POST/api/datahandlers/{datahandler_id}/resetkeyadmin
POST/webhook/{datahandler_id}/{datahandler_key}public

In the first phrase on the endpoints in italic will be implemented.

Project’s resources

The forget-me app has been open sourced under AGPL v3 and it’s published to GitHub. Should you encounter an issue or have an idea, submit a ticket to JIRA.

Next in this series

GDPR compliant forget-me app with Spring Integration (Part 2): In and outbound messaging

This second part focuses on how to define in- and outbound messaging with Spring Integration’s AMQP support by using the new Java DSL, which is now (as of version 5) part of the core project and doesn’t have be included as a separate dependency.

GDPR compliant forget-me app with Spring Integration (Part 3): Conditional configuration with Spring Boot 2

Altought there will be only a single adapter supporting MailerLite, the app is modular in design and adapters are activated dynamically once they’re configured.

GDPR compliant forget-me app with Spring Integration (Part 3): OAuth2 login

Spring Security 5’s brand new OAuth2 support is used and in this part we’ll look into how the login flow can be customized.

Laszlo Csontos
 

I've been coding since the age of 9. I knew from childhood that all I wanted to do was code. Now I've been coding for 25 years, with Java for 18 years and professionally for 13 years. During past projects I worked in various roles as a consultant, developer, mentor, team leader and architect. My focus areas have been database- oriented back-end applications, performance tuning techniques and distributed systems. In the last 3 years, I specialized in building microservices with the Spring Ecosystem and also contributed to some of its sub-projects. The newest venture of mine is the creation of springuni.com, which aims at helping young software engineers learn Spring.