Editing Finding relations between variables in time series
Jump to navigation
Jump to search
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
{{Topic Infobox}} | {{Topic Infobox}} | ||
− | + | A frequent need when engaging in personal science is finding relationships between different variables across a time series, an example could be the question "does eating chocolate improve focus?". | |
− | + | == How does one find relations between variables == | |
+ | To do this you need to have your data parsed, cleaned, all in one place, and ideally even visualized. The next step is to find relations between variables. Some people do this by themselves, by using programming languages such as R and Python in notebooks or apps. Coding platforms such as the notebooks on [[Open Humans]], Kaggle or GitHub can help, but it frequently | ||
+ | requires technical skills. | ||
− | + | There are also a number of tools or apps that can semi-automatically perform these correlations and help in doing the analyses. | |
− | |||
− | |||
== List of less technical tools == | == List of less technical tools == | ||
− | |||
==== [[Open Humans]] and their Personal Analysis notebooks ==== | ==== [[Open Humans]] and their Personal Analysis notebooks ==== | ||
Line 16: | Line 15: | ||
==== [[Zenobase]] ==== | ==== [[Zenobase]] ==== | ||
[https://blog.zenobase.com/post/81497604762 Zenobase] can test correlations based on user-specified questions. User must configure lag, regression method and aggregation method using a UI. Powerful filtering tools too. | [https://blog.zenobase.com/post/81497604762 Zenobase] can test correlations based on user-specified questions. User must configure lag, regression method and aggregation method using a UI. Powerful filtering tools too. | ||
− | |||
− | |||
− | |||
==== Data Flexor ==== | ==== Data Flexor ==== | ||
Line 43: | Line 39: | ||
==== young.ai and [http://www.aging.ai/ aging.ai] ==== | ==== young.ai and [http://www.aging.ai/ aging.ai] ==== | ||
− | + | deep learning predictor of age based on human blood tests and young.ai makes recommendations | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
[[Gyroscope]] | [[Gyroscope]] | ||
Line 64: | Line 54: | ||
Wellness FX | Wellness FX | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
== List of very technical tools == | == List of very technical tools == | ||
− | |||
− | |||
Programming languages for statistics; Matlab, R, Python, Julia. | Programming languages for statistics; Matlab, R, Python, Julia. | ||
Line 82: | Line 62: | ||
==== [[List of Interesting Self-Tracking Results#Observational.2C%20Many%20variables|DIY Individuals]] ==== | ==== [[List of Interesting Self-Tracking Results#Observational.2C%20Many%20variables|DIY Individuals]] ==== | ||
+ | Some people allow people to use their scripts that analyze lots of data at once but this does require some programming skill. | ||
+ | |||
== Reasons time series analysis especially as applied to QS is hard == | == Reasons time series analysis especially as applied to QS is hard == | ||
[https://forum.quantifiedself.com/t/my-baseline-network-physiology-10-days-of-eeg-egg-ekg-cgm-temperature-activity-and-food-logs/5671/19 Wavelet coherence] is one potential solution. | [https://forum.quantifiedself.com/t/my-baseline-network-physiology-10-days-of-eeg-egg-ekg-cgm-temperature-activity-and-food-logs/5671/19 Wavelet coherence] is one potential solution. | ||
− | + | [http://www.tylervigen.com/spurious-correlations Spurious Correlations] mostly shows that if two things are trending in one direction and are checked for correlation they will show a very significant correlation. Practice effect is a subset. Another is one instance of an event increases the chances of the same event happening soon after. Economists suggest unit root. | |
− | |||
− | [http://www.tylervigen.com/spurious-correlations Spurious Correlations] mostly shows that if two things are trending in one direction and are checked for correlation they will show a very significant correlation. Practice effect is a subset. Another is one instance of an event | ||
− | |||
− | |||
Lag. What if eating pizza on one day causes heartburn the next? | Lag. What if eating pizza on one day causes heartburn the next? | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
Few positive instances but they are important. Went to a specific restaurant twice got sick soon after twice. Only ever got sick with similar symptoms five times. Or. Two large rare humps happen almost one after the other, similar to previous example if treated as events, adding the fact that lots of samples showing their similarity in shape too. | Few positive instances but they are important. Went to a specific restaurant twice got sick soon after twice. Only ever got sick with similar symptoms five times. Or. Two large rare humps happen almost one after the other, similar to previous example if treated as events, adding the fact that lots of samples showing their similarity in shape too. | ||
− | Since removing | + | Different sampling rates need to be interpolated to be compared. Window. Since removing the effects of other variables makes the variable of interest's effect stand out, machine learning must be used. Common approach would be to bin predictor variables multiple ways based on time from effect being checked, mean or other aggregator method and window of the aggregator. |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | Machine learning also has limits on the kind of patters it can detect. | |
− | + | Types of data. [Exercised] is an event with specific occurrence moment and length while [tired] is a vaguer value user could use to try to describe feelings past 4 hours. | |
− | |||
− | |||
[[Category:Data analysis]] | [[Category:Data analysis]] |