Changes

Most personal science projects require finding relationships between different variables of the type 'time series'<ref>Core-Guide_Longitudinal-Data-Analysis_10-05-17.pdf (duke.edu)</ref>. An example could be the question "does my daily chocolate consumption correlate with my daily focus score?".   

Most personal science projects require finding relationships between different variables of the type 'time series'<ref>Core-Guide_Longitudinal-Data-Analysis_10-05-17.pdf (duke.edu)</ref>. An example could be the question "does my daily chocolate consumption correlate with my daily focus score?".   

You could do experiments if you control everything rigidly or if the effects are strong and quick, like less than a week. Old data may be useable as Baseline.  

You could do experiments if you control everything rigidly or if the effects are strong and quick, like less than a week. Old data may be useable as Baseline and a baseline may rule out some issues. If both block (like 2 weeks) and daily mixed (random intervention every day)  produce the same results then issues of time series are probably not in your experiment.   

Finding more complicated relationships require better statistical tests and algorithms and data science skills. Apps that would do this automatically or at least easily are not yet available. See below. Most internet resources treat time series as (regular cyclical) series, which is not useful as most of the tracked variables have irregular patterns and don't even have a regularly cyclical component.   

Finding more complicated relationships require better statistical tests and algorithms and data science skills. Apps that would do this automatically or at least easily are not yet available. See below. Most internet resources treat time series as (regular cyclical) series, which is not useful as most of the tracked variables have irregular patterns and don't even have a regularly cyclical component.   

==== Sonar [https://www.sonarhealth.co sonarhealth.co] ====

==== Sonar [https://www.sonarhealth.co sonarhealth.co] ====

Customizable aggregation and syncing like weigh fitbit twice as much as apple watch or average steps instead of sum.

Customizable aggregation and syncing like weigh fitbit twice as much as apple watch or average steps instead of sum.

[[Gyroscope]]

[[Gyroscope]]

Export from Apple Health<ref>github.com/Lybron/health-auto-export</ref> (no analysis)

Export from Apple Health<ref>github.com/Lybron/health-auto-export</ref> (no analysis)

== List of very technical tools ==

== List of very technical tools ==

Really strong relationships will be detected even through most of these problems.  

Really strong relationships will be detected even through most of these problems.  

[http://www.tylervigen.com/spurious-correlations Spurious Correlations] mostly shows that if two things are trending in one direction and are checked for correlation they will show a very significant correlation. Practice effect is a subset. Another is one instance of an event increases the chances of the same event happening soon after. Economists suggest unit root.

[http://www.tylervigen.com/spurious-correlations Spurious Correlations] mostly shows that if two things are trending in one direction and are checked for correlation they will show a very significant correlation. Practice effect is a subset. Another is one instance of an event type A increases the chances of the same event type happening soon after. Economists suggest unit root.

Effects on target variable from outside known variables. In non time series this is compensated for with RCT but in time series such an effect may last a while and coincide with an intervention causing very false results. This problem makes baseline data gathering more difficult and also necessary. Sometimes a baseline will show that this issue does not occur for a particular target variable. Alternatively experimenter could compensate by strictly controlling all possible sources of variance.   

Effects on target variable from outside known variables. In non time series this is compensated for with RCT but in time series such an effect may last a while and coincide with an intervention causing very false results. This problem makes baseline data gathering more difficult and also necessary. Sometimes a baseline will show that this issue does not occur for a particular target variable. Alternatively experimenter could compensate by strictly controlling all possible sources of variance.   

Build up. What if it takes two days of eating pizza to cause heartburn?  

Build up. What if it takes two days of eating pizza to cause heartburn?  

Few positive instances but they are important. Went to a specific restaurant twice got sick soon after twice. Only ever got sick with similar symptoms five times. Or. Two large rare humps happen almost one after the other, similar to previous example if treated as events, adding the fact that lots of samples showing their similarity in shape too.  

 

Bin. Window. Smooth. Variables only make domain sense as aggregate over some time. Variables have a really high sampling rate.

 

Interpolate. Variables have different sampling rates so need to be interpolated to be compared.

Different sampling rates need to be interpolated to be compared. Window.

Types of data. [Exercised] is an event with specific occurrence moment and length while [tired] is a vaguer value user could use to try to describe feelings past 4 hours.  

Since removing real effects of other variables on target variable makes the variable of interest's effect stand out, machine learning will be used. Basic approach would be to bin predictor variables multiple ways based on time from effect being checked, mean or other aggregator method and window of the aggregator.  

All the [[Issues with Self Report]] .    

Machine learning also has limits on the kind of patterns it can detect.  

Few positive instances but they are important. Went to a specific restaurant twice got sick soon after twice. Only ever got sick with similar symptoms five times. Or. Two large rare humps happen almost one after the other, similar to previous example if treated as events, adding the fact that lots of samples showing their similarity in shape too.  

Types of data. [Exercised] is an event with specific occurrence moment and length while [tired] is a vaguer value user could use to try to describe feelings past 4 hours.    

Since removing real effects of other variables on target variable makes the variable of interest's effect stand out, 'machine learning' needs to be used. Basic approach would be to bin predictor variables multiple ways based on time from effect being checked, mean or other aggregator method and window of the aggregator.  

All the [[Issues with Self Report]] .    

Machine learning also has limits on the kind of patterns it can detect.  

=== What to expect from the complete analysis tool ===

=== What to expect from the complete analysis tool ===

Cycles decomposition using a model like ARIMA. Ex. kayak season is in the summer or lunch is at exactly 1pm.   

Cycles decomposition using a model like ARIMA. Ex. kayak season is in the summer or lunch is at exactly 1pm.   

Detection of repeated shapes implying similar events that are not cyclical; like dinner is anywhere between 4pm and 10pm and causes a particular 2 hour spike in glucose.

Detection of repeated shapes implying similar events that are not cyclical; like dinner is anywhere between 4pm and 10pm and causes a particular 2 hour spike in glucose.

== References ==

== References ==

[[Category:Data analysis]]

[[Category:Data analysis]]