How long until a personal pattern is real?
A simulation study found about a fifth of one person's real links at 75 to 100 measurements, half at 500. What it means for you, and how Lomba waits.
By Lomba 4 min read
Start logging a habit, and within two weeks something will look like a pattern: a higher recovery score after the sauna, say. A 2023 study in Psychological Methods asked how many measurements one person needs before the links in their own record can be found.
We read it, then put the same question to Lomba's own check.
What the study did
- Who
- Five researchers at the University of Amsterdam. Nobody was tracked for it. It is a computer simulation built on two earlier diary data sets.
- What was compared
- Records of 75, 100, 500 and 1,000 measurements from one simulated person, with 6 to 12 things tracked at once.
- What was measured
- How many of the real links between the tracked things a standard method found, and how often it reported a link that was not there.
- Who paid
- A grant from the Centre for Urban Mental Health at the University of Amsterdam. The authors declare no conflict of interest. One is an associate editor of the journal.
Four takeaways
At 75 to 100 measurements, about a fifth of the real links were found.
With six things tracked, the median was 19% at 75 measurements and 23% at 100. The authors call about 100 the most that is usually feasible, for example five entries a day for 3 weeks.
Five times the data found about half.
At 500 measurements the median was 48%. In the authors' words, "we can, on average, only detect 50% of the total number of edges". An edge is a link.
What it did find was rarely false.
The strongest links were the most likely to be found. On a short record, a careful method mostly stays silent.
Fewer things tracked, more of the links found.
With 75 to 100 measurements the authors advise tracking about six things. They add that a record showing few links "should not be interpreted as an actual lack of connections".
A full line is every real link found.
From 75 to 500 measurements, the share of real links found went from 19% to 48%. Pick a row for what that many measurements means.
The main takeaway
One person's record is small. A real link needs many ordinary days before it can be told from chance.
A smaller link needs more of them. That means weeks before a first look and months for most answers.
So a gap that shows up in the first weeks is not yet a pattern. Track enough things and some gaps appear by luck alone, and a short record cannot tell those from a real one. The honest early answer is "not yet".
What Lomba does with that
Lomba waits until the days are there. Until there are 5 days with the habit and 5 without, the answer is "Still collecting" and no gap is shown.
To test a gap against chance, Lomba's rule asks for more: at least 8 days each way and, for the morning measures a wearable fills in, habit days in at least 4 different weeks and 40 mornings in all. It needs ordinary days with and without the habit, not days arranged for it.
A full line is 12 weeks of logging.
A habit done twice a week: no gap is shown before day 18, and a gap can first be tested against chance on day 40.
Then Lomba shuffles which mornings count as "after" at least 2,000 times, keeping neighboring days together, and counts how often a gap that large turns up anyway. The bar rises with every question asked: put 100 questions to a record with no real link, and a plain 1-in-20 test passes about 5 of them by luck. To be called Held up, a gap must also pass further checks, such as coming back over at least 6 separate runs of days. How it works follows two examples through them.
In Lomba's own simulation, a large planted gap reached Held up in 54% of made-up records after 90 days of logging and in 91% after 180. The assumptions: 120 records for each gap size, each with 18 habits, 60 days of wearable history before logging starts and one real gap planted. Large means 1 standard deviation, about 2 beats per minute of resting heart rate in those records. In Lomba's own simulation a gap half that size reached Held up in 4% of records after 90 days, 25% after 180 and 50% after a year.
It has limits, and we would rather say them:
- Simulated, not people. The study's records were built on mood and behavior diaries, where one measurement is one diary entry, not one day. Lomba's own simulation uses made-up records, and a real record can be faster or slower.
- Checking again adds chances. Lomba runs its check again as days arrive, and each run is another chance for luck to pass.
- A pattern, not a cause. A gap that holds up is a pattern in your own days, not proof of cause, and not medical advice.
Sources
- Mansueto AC, Wiers RW, van Weert JCM, Schouten BC, Epskamp S. Investigating the feasibility of idiographic network models. Psychological Methods, 2023.
- The same paper, free to read in the University of Amsterdam's repository. Final published version.
The simulation numbers under "What Lomba does with that" are Lomba's own, on made-up records. They are not published elsewhere.