If some doctor by luck read this, please check if the above is healthy. It is time spent in each heart rate zone per day during the last 3 months. I’ve filtered out sleep and exercise to have it include only resting heart rate.

Health Data

Apple Health App is recording my heart rate, steps, workouts, stand hours, sleep etc. All data can easily be exported as a zipped xml-file.

Since F# have a good interactive repl and some interesting data science libraries (eg. FsLab and Deedle) I decided to use it to experiment with the data.

Source code for experiments: https://github.com/henrikwallstrom/fsharp_apple_healthdata

My first failed attempt was to use F# Data XmlProvider to read the XML-file.

I’m not sure if it’s the structure or size of the file but it was way too slow.

I instead went for reading XML manually which is quite pleasant in F#.

The DOCTYPE declaration of the file include an internal document type definition that describes the structure of the XML document and element types that exists.

It looks like the most interesting part is the Record elements.

The document consists of a very long list of health record nodes that can be matched with the XPath expression /HealthData/Record

Almost one million records after 3 months.

Health Records

It looks like records data have type, unit, sourceName, startDate and endDate in common.

We can make a pass over all health records and check what types that exists.

It would be nice to plot that as a PIE chart.

FsLab and XPlot

FsLab is a collection of libraries for data-science. Among many things it includes XPlot. XPlot is a cross-platform data visualization package for F# powered by Google Charts and Plotly.

XPlot Documentation: http://tahahachana.github.io/XPlot/

Record types as a pie chart:

I will focus on HKQuantityTypeIdentifierHeartRate, HKQuantityTypeIdentifierAppleExerciseTime and HKCategoryTypeIdentifierSleepAnalysis to get heart rate readings when not sleeping or exercising.

Parse records

We could parse the XML elements into a list of dictionaries

We could then use that and plain F# to get quite far but I would like to try Deedle.

Deedle

Deedle is a library for data and time series manipulation. There are two basic concepts, Series and Frames. A Series is a vector like collection of values indexed by a key. We will mainly work with time series where the keys are time stamps. A Frame is a matrix like collection of series that share the same row key. We will use frames to e.g. group properties of health records.

A frame can be constructed in a few ways. One way is using Frame.ofValues from a sequence of tuples:

It takes tuples of row * column * value. We could turn our records into tuples like that by mapping over all Records and use the row index as row in the tuple. Then map over all entries in the dictionary in each row and use dictionary key as column and value as value.

Heart records

Heart Rate records/samples are collected by my Apple Watch and Chest Strap. I suspect there are some overlaps between the two sources.

Unit seems to be count/min on all records.

Start and end dates are the same on most records so I guess the value is for a given moment. Some HR entries seems to have a duration. I guess it is a bug.

As I suspected there are duplicate HR readings with the same start and end date:

But since I have only one heart I guess I can pick any of the readings and filter with Seq.distinctBy.

Value is a string. We can add the parsed number value as a new column.

Heart rate readings are read more often when eg. exercising or using the chest strap we therefore give them a weight by calculating the duration between readings.

The watch usually read every 5th minute so let´s assume that gaps larger than 5 minutes (300s) are periods without readings

Most readings are done every 10th second or often since the watch read every 5th minute during rest but more often when exercising.

Before playing more with HR we need to add sleep, standing and exercise to the Frame

Sleep

Information is stored when I go to bed, wake up and actually sleep.

Sleep readings from different sources can overlap.

Merge records

Metrics such as sleep, exercise, stand can overlap when collected from different sources.

We could:

• Sort records by start date
• Merge records with previous record if start is before the previous end

We can merge two records by mapping over the first records values, keeping values that does not match the new field and emitting a new value for the updated field.

We can now set endDate:

Now we can merge overlapping records by sorting the list and merging records with the previous record if they overlap.

Example:

Back to sleep

Sleep recorded by different apps and merged:

Calculate sleep duration:

Histogram of the durations:

Looks like nights for a parent of a 3-year-old.

Merge in activities

For every time stamp in HR readings I would like to know if I was awake, exercising or standing. I want a IsASleep, IsExercising and IsStanding column/flag on each HR reading row.

Since activity frames have a duration (start - end) while hr readings are momental (timestamp) we need to first transform them.

We need to split rows of durations:

Into timestamp rows:

Frame to Time Stamp Frame:

Example:

Deedle can merge frames. We could eg. join heartRateFrame with sleepRecordsFrame like this:

It would give us a new frame with all HR readings and a sleep flag when there is a sleep row that exactly match the key/startDate or if there is a smaller/previous key.

Let’s say I was a sleep at 23:30 and have a HR reading 23:31 I can assume I was still a asleep

Looks like I’m sleep walking :)

Resting HR

I want my resting heart rate. We can do that by filtering out sleep and exercise.

But it looks like I have HR readings above 110 with time stamps within 10s. I assume that is exercise that is not logged correctly. I will filter them out as well.

Average resting HR per day:

HR Zones

I want a stacked column chart with %-of-minutes-of-the-day in different HR zones per day.

I will use the same colors and HR zones as the iOS app Heart Watch. It is a great iOS app that you should buy.

Helpers to extract data from table:

We can construct a Pivot Table with:

• Dates as rows
• HR zone name as columns
• Total minutes in HR zone as cell values

And plot the table as a stepped area chart:

A problem with the above chart is that I don’t have the same number of readings every day since I don’t wear it all hours and also exercise less some days.

We can get a table with the total logged resting HR minutes per day.

Then apply it to our previous values to get them in % of the daily totals:

Deedle allow mathematical operations to be applied to series similar to matrix operations. Deedle automatically aligns the series and applies the operation on corresponding elements.

Zone minutes in % per day:

Voila! We are DONE!!!

In 2011 we decided to build our own JS-framework, WIDGET-JS. I would probably not have done that today with all options available.

But it is not that complicated to do and it would be interesting to re-do it in ES6/ES20015. Let´s reimplement the core of Widget-JS for fun!

We want to be able to write code like this:

My post is more or less a transcript on how to get to this result: https://gist.github.com/henrikwallstrom/12669c3865c9b90f07b3.

ES6/ES2015

ES20015 features we will use:

• Let declarations for block scoped variables. let x = {}

• Destructuring assignment for eg. assignments let { width, height } = getDimensions() and function parameters function fun({name, age})

• Spread syntax for functions that take multiple arguments when we have eg. an array. fun(...someArray);

• Rest parameters for variable number of arguments to functions. function fun(id, …args) {}

• Arrow functions (a, b) => a + b

We will not use the new class syntax mainly because of personal taste and that it still have issues with how “this” is bound. Instead we will use factory functions and closures. If you are interested in why people consider the new classes bad go to eg. https://github.com/joshburgess/not-awesome-es6-classes

Rendering DSL

Lets start with a htmlBuilder. Maybe it should be named domBuilder since we will not generate HTML. We will append DOM elements.

Our builder take a rootElement as argument and appends child nodes to it from tag names. Now we can create tags like this:

Wohoo!! We can create horizontal rulers, forced line-breaks and tags that don’t need children or attributes. Maybe not that awesome? We need to be able to nest elements:

Let’s create a htmlTag for that. Our htmlTag? will wrap a DOM element and expose methods to append children or mutate the element.

We now have a tag that we can append text, elements and other tags on. Also anything that implements the appendToTag interface. Let’s update our builder to use tags:

We can now nest tags:

It is starting to look like our rendering DSL. But we want to write html.h1() instead of html.tag(‘h1’). So lets define all the common tags:

And add them to our builder:

And now we have our Rendering DSL.

But we need to be able to add attributes, eg. by allowing an object with key/values to be passed as an argument like this:

We can do that by assuming an object without appendToTag to hold attributes:

And add a that.attr that set attributes for all key/value-pairs:

Voila!

Let´s attach functions!

Now we can listen for eg. clicks

jQuery or no jQuery

It’s easy to add support for jQuery. Just add one line to htmlTag:

Use it like:

But you don’t like jQuery? You might not need it http://youmightnotneedjquery.com/

Just add what you need to htmlTag

Stateful components aka widgets

We already have what we need to create a stateful component that keep its state and re-render itself when state change. Eg. this counter widget with two buttons to increase/decrease the count

And we have a reusable counter widget that we can create instances of like this:

But we don’t want to write all of that boilerplate code for every widget. Let’s extract the update mechanism and common code into a widget class:

Now we can write our counter widget as:

Composable components

Reusing and composing widgets gives us the real power. Since widget have the method that.appendToTag widgets can be appended to tags.

Eg. to make an unordered list of users we do:

ES6 or not?

I wish JavaScript would have had all of these nice ES6 features from the start. It make the code a lot more compact and clean. But I’m not sure it is worth the extra transpile step. I think we will wait until mainstream browsers support ES6 fully. Then we still need transpillation for older browsers but can develop without transpilation.

So, during my Christmas vacation I decided to fix that by writing modules to manage IIS web sites, web applications, virtual directories, application pools and web bindings`.