From local storage To-do list to Fractal, Peer-to-Peer Habit Tracker

Part 2: Multilayered architecture

(Part 1)

While I was studying the LS185 course in the Launch School curriculum, I had the idea for my own web app — a habit tracker that would allow you to dissect patterns of behavior into their smallest constituent parts. I thought of the inital idea after being exposed to the book ‘Atomic Habits’ by James Clear, and using the WorkFlowy application.

The difference with my application compared to other linear habit trackers like HabitBull, or the simple crossing off of calendar days, was that I was going to make a *fractal* way of visualizing behaviour (the concept is succinctly explained here by a fellow Medium writer).

I also had ideas about changing the core architecture to a decentralised ‘community network’ allowing learning from each others’ habits but with sovereignty of personal data. The discovery of Holochain — an emerging decentralised framework & cloud computing infrastructure — was what got me into serious backend coding from the start. For me this would be the long term goal — make my first Holochain app. But first I would explore client-server architecture in-depth and make a single paged application.

Learning from the first attempt

The local storage version I had already made was based quite closely around the Todo list app that I had followed along with in the LS curriculum. We were advised to experiment and make our own simple applications in this fashion.

I showed the app in its prototype state to a fellow student on the course, Steve. He had more JavaScript experience than me and saw some of the limitations of my design. I had become somewhat attached to using the jQuery Swiper component and it was leading me to write lots of hard-to-maintain code.

Now I was starting to learn that there are tools out there for almost every job, and thankfully Steve pointed me towards the library D3js. I was still somewhat new at modular design and not used to incorporating anything other than jQuery into my approach to manipulating the DOM.

Planning a new UI

I spent some time getting to grips with the basics of the D3 data join, different layouts, and how I could visualise my fractal structure in the browser. When developing the frontend visualisations:
1. I would need to incorporate d3-zoom.
2. The closest layout that I could envisage my data structure working with was a zoomable icicle diagram where only 3 levels are displayed at any one point.
3. This is based on a hierarchical data structure so I needed to explore the way that d3 works with hierarchical data.
4. To try and load a full tree of data for each user at once would cause issues rendering, once the tree reached a certain size.
5. I could instead populate the data join with the level of the hierarchy that I wanted to visualise at any one moment (a subtree), and query the ‘edges of my data’ (the leaves of the currently displayed tree) at the specific level I was viewing. The idea of pagination data being fetched in increments seemed to be the closest approximation.

Backend SOLID principles

In the period between first design and sketching the above path to a MVP, I also studied relational databases using PostgreSQL. I had always known that local storage would be good for nothing other than a personal prototype of the interface and concept, but now I was faced with the task of choosing a persistent data store. My main challenge from Part 1was how to separate concerns —as Uncle Bob had said — to delay making that decision final for as long as possible.

It took me a little time to get my head around dependency injection & inversion of control, as well as separating out all responsibilities for Object Relational Mapping. I spent time learning the dry-rb libraries and their uses. The libraries make Uncle Bob’s vision easier to implement!

Choosing a stack

I used a starter for a spike, just to explore the multi-layered SPA structure.

I had been reading about the ‘Build First’ philosophy of development workflow, asset bundling in JS, and modular design. All of this would need to be incorporated going forward. I had started to learn about the use of reactive JS frameworks but not wanting to deviate too far from the Launch School philosophy of ‘mastery of fundamentals’, I decided not to learn any big framework, but picked up MithrilJS (it’s lightweight and easy to get started with).

BACKEND:
- Sinatra for routing
- Sequel and ROM-rb for ORM
- dry-rb, a set of modern Ruby libraries, for code organisation and DI
- PostgreSQL

FRONTEND:
- MithrilJS
- D3 for visualisation
- Axios to query the backend
- Webpack

I would set up both layers as individual Docker containers and allow continuous intergration/development with Github actions by pushing new versions of the images to Docker Hub with each push to the Main git branch.

What I came up with

I made a web API for storing and serving hierarchically structured (JSON) serialised habit data. The design choices I took enable rapid interaction between the application layer and the data persistence layer, and the ability to ‘switch out’ the ‘Source of Ultimate Truth’ from a database to a file of serialised data, simply by passing a query string `?demo=true` to the frontend. I managed to do this while retaining most of the core functionality of the app (vs. when it was using the PostgreSQL persistence layer).

My initial objective was to demonstrate Uncle Bob’s premise of SOLID design allowing you to make the database ‘an implementation detail’. I was hoping that this would allow adaptability towards decentralised architecture in the future.

The library ROM-rb (Ruby Object Mapper), with its YAML adapter, was crucial to this, but the adapter does not allow writing to files. This meant that I could only make a ‘read-only’ version that would not persist changes to the serialised data for longer than the session. For demonstration purposes, this is fine, and leaves the option for easily swapping back the persistence layer to a database after playing around with the structure of the web scraped data: an adapter in the true sense. This will in turn make the app more immediately accessible as people will be able to ‘import’ web-scraped habit structures and then make them their own by persisting to the database.

EDIT: After migrating the UI to React, the demo of this feature is no longer live. I have instead implemented a UI demo with pre-built templates to play around with.

Areas for Improvement

Due to the fact that I had yet to discover React & Redux at the time, I didn’t come up with a cohesive state management pattern for the frontend. By the time I was using multiple levels of component state this become a real issue. I later learnt that the Meiosis pattern would have gone very well with MithrilJS’s lightweight approach and it is a good introduction to Redux’s patterns.

Since starting to build the Holochain-backend version I have focused on Redux from the start and expect to rewrite many of the frontend components to work with React, which will make the fractal view much more responsive and smooth out the state management bugs mentioned above.

How many of my 5 UI points did I manage to hit? It is currently a zoomable tree diagram, without the buffered ‘pagination’ type feed of data. The React version should hit all 5 to make the visualisation fractal.

I am pleased with the amount I learnt through this primarily backend coding journey. How far can we take the Todo list?

Find out in part 3!