```custom-WatchMotionIndex
activePost: 4
```

![{2400[x]1318}](/journal/241124/watch-motion-a-journey-through-raw-data-1.jpg)

Spent a couple of hours at my favorite coffee shop this weekend to do some exploratory data analysis of the data collected so far.

Let's open up a Jupyter Notebook and dig into the data! First I list the contents of the bucket and print a couple of sample files for each exercise so we have something to work with:

```markdown
+--------------+-----------------+------------------------------------------------+
| Exercise     | Number of Files | Sample Files                                   |
+--------------+-----------------+------------------------------------------------+
| battlerope   | 49              | 27PXR, 3S3Y9, 3ZYW5, 5A479, 5FLRB (+44 more)   |
| chins        | 54              | 2AJNF, 2JGC2, 2QD55, 3CQGL, 4UJBZ (+49 more)   |
| press        | 51              | 2XQWW, 2ZXJC, 39YPF, 3YXKX, 498DH (+46 more)   |
| situps       | 55              | 29V2P, 2Q9JU, 2SXVW, 4GNA3, 4PFVJ (+50 more)   |
+--------------+-----------------+------------------------------------------------+
```

Here's a sample of what each file looks like:

```csv
acc_x,acc_y,acc_z,gyro_x,gyro_y,gyro_z,mag_x,mag_y,mag_z
-0.1658,0.08798,-0.08062,0.3743,-0.8699,-0.4627,37.865,-11.728,-33.783
-0.1332,0.0951,-0.01393,0.4457,-0.9735,-0.5086,37.844,-11.771,-33.803
-0.07474,0.004615,-0.01889,0.3523,-1.0037,-0.4846,37.893,-11.788,-34.096
-0.02115,-0.07249,0.08809,0.05389,-1.0022,-0.5502,37.731,-12.035,-34.409
...
(1000 rows)
```

We'll look at a situps exercise since I believe those will have pretty clear patterns. Picking a random one (2Q9JU if you're curious) and plotting the different sensors looks like this:

![{2400[x]1318}](/journal/241124/watch-motion-a-journey-through-raw-data-2.png)

If we take a closer look at the data from the magnetometer, we can see that the first 100 or so samples are much noisier than the rest, probably due to sensor initialization and calibration. To remove this noise, we can simply discard the first 100 samples.

Let's also try to see what happens if we scale each column to have a mean of 0 and a standard deviation of 1:

![{2400[x]1318}](/journal/241124/watch-motion-a-journey-through-raw-data-3.png)

Next, let's plot a heatmap over the values to get a feel for the correlations between the different sensors:

![{2400[x]1318}](/journal/241124/watch-motion-a-journey-through-raw-data-5.png)

The data is pretty noisy so we will apply a moving average to smooth it out, here's an example taking a window size of 100 samples:

![{2400[x]1318}](/journal/241124/watch-motion-a-journey-through-raw-data-4.png)

We of course lost some details, but we can now see the general trend of the exercise. Let's also try to do a 3D plot of the data to see if we can see any interesting patterns:

![{2400[x]800}](/journal/241124/watch-motion-a-journey-through-raw-data-6.png)

Also a bit noisy, but we can apply a moving average to smooth it out, here's another example with a window size of 100 samples:

![{2400[x]800}](/journal/241124/watch-motion-a-journey-through-raw-data-7.png)

Very clear pattern for all sensors, especially if we disregard the X-axis drift of the magnetometer.

With this data, we should be able to detect peaks, valleys and other interesting features. Let's count the number of peaks for each sensor:

![{2400[x]1318}](/journal/241124/watch-motion-a-journey-through-raw-data-8.png)

```markdown
+-----------------+----------------------+
| Column          | Number of Peaks      |
+-----------------+----------------------+
| acc_x           | 7                    |
| acc_y           | 6                    |
| acc_z           | 6                    |
| gyro_x          | 5                    |
| gyro_y          | 5                    |
| gyro_z          | 6                    |
| mag_x           | 6                    |
| mag_y           | 6                    |
| mag_z           | 6                    |
+-----------------+----------------------+
```

I'm using the `find_peaks` function from the scipy library and it works surprisingly well. The signature looks like
```python
find_peaks(x,
           height=None,
           threshold=None,
           distance=None,
           prominence=None,
           width=None,
           wlen=None,
           rel_height=0.5,
           plateau_size=None)
```

Using a `prominance` of 0.4 and a `distance` of 50 samples (expecting each exercise to take at least 0.5 seconds). One idea could be to just take the mean of the number of peaks for each column and let that be the number of repetitions. Remains to be seen how this approach generalizes to other exercises though.



And that's what we'll look at next!

A journey through raw data

Watch Motion

```custom-WatchMotionIndex
activePost: 3
```

I'm still collecting data for the model, but I thought I should try to do some on-device inference to get a feel for how it performs and if it's fast enough. I used the training data I have collected so far and training a very naive model without much preprocessing. The model is trained on a window of 500 samples (5 seconds at 100Hz), and I run the inference every 100 samples (using the latest 500 samples).

<iframe
  className="video-1-2 mb-8"
  src="https://www.youtube.com/embed/dV49P3sZiJg?si=iq54NC3G1wdatGE3"
  title="YouTube video player"
  allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
  allowFullScreen
></iframe>

To get started, I downloaded the data from DynamoDB with a simple Python-script and saved it to disk as JSON files. I then did some very simple preprocessing, just normalizing the data and splitting it into training and validation sets. I'm using CreateML to train the model, it's great for getting started but gives very little control over the training process so I will probably switch to TensorFlow or PyTorch later. This is what it looks like after training for 20 epochs:

![{2400[x]1318}](/journal/241117/watch-motion-first-try-with-on-device-inference-2.png)


First try with on-device inference

![{2400[x]1600}{priority}](/journal/241116/infinite-mass-at-pustervik-1.jpg)

![{2400[x]1600}](/journal/241116/infinite-mass-at-pustervik-2.jpg)

![{2400[x]1600}](/journal/241116/infinite-mass-at-pustervik-3.jpg)

![{2400[x]1600}](/journal/241116/infinite-mass-at-pustervik-4.jpg)

More photos and prints at [Rockfoto](https://rockfoto.nu/artists/infinite-mass/20241116/30181)

Read the review at [GP.se](https://www.gp.se/kultur/musik/konsertrecension/recension-infinite-mass-pustervik-lordag.07ad24d1-42ff-4fdd-9708-0efec8f8bdd6)


Watch Motion

First try with on-device inference

Project: Watch Motion