Editing Data format structure variables
Jump to navigation
Jump to search
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
− | + | This page is mainly for DIYers with at least some knowledge of [[Excel]] or other spreadsheet software. | |
− | This page is mainly for DIYers with at least some knowledge of [[Excel]] or other | ||
==== File Format ==== | ==== File Format ==== | ||
Line 6: | Line 5: | ||
==== Structure ==== | ==== Structure ==== | ||
− | Statistical analysis of self tracking data is usually done on tabular data, like spreadsheets, with rows representing individual observations. | + | Statistical analysis of self tracking data is usually done on tabular data, like spreadsheets, with rows representing individual observations.<ref>https://en.wikipedia.org/wiki/Relational_database</ref> In all but a few cases this is sufficient structure. |
Line 19: | Line 18: | ||
− | States that are written once and apply until changed to something else. For example, place of residence or whether a brace is being worn continuously. This structure is similar to a simple "event" with just a | + | States that are written once and apply until changed to something else. For example, place of residence or whether a brace is being worn continuously. This structure is similar to a simple "event" with just a when-time and what though duration is calculated from replacement. |
Line 25: | Line 24: | ||
− | [[Tools for journaling, thoughts and note taking|Journal]] entries and notes. Often journal entries are written texts describing the day. | + | [[Tools for journaling, thoughts and note taking|Journal]] entries and notes. Often journal entries are written texts describing the day. |
==== Variables ==== | ==== Variables ==== | ||
Line 33: | Line 32: | ||
* A variable that depends on previous values of this same variable is not independent and is called auto-correlative? and non-stationary. For example skills at playing the guitar. | * A variable that depends on previous values of this same variable is not independent and is called auto-correlative? and non-stationary. For example skills at playing the guitar. | ||
* Randomness of Missingness. Similar to independence but its not the value of the variable but whether other measured variables could correlate with higher incidence of missing values. For example forgetting to charge the smart band because of tiredness and having a night without it on. | * Randomness of Missingness. Similar to independence but its not the value of the variable but whether other measured variables could correlate with higher incidence of missing values. For example forgetting to charge the smart band because of tiredness and having a night without it on. | ||
− | * Target. Level. Is this variable something you want to improve, or a variable likely to affect those | + | * Target. Level. Is this variable something you want to improve, or a variable likely to affect those or just an intermediary background variable measured because it was easy and provided context? |
* Similarity. Proxy. Is this variable measuring something very similar to what another variable is measuring. The most common example is [[Tools for heart rate or pulse|heart rate]] as many wearable measure it and the avid self tracker always has a few. | * Similarity. Proxy. Is this variable measuring something very similar to what another variable is measuring. The most common example is [[Tools for heart rate or pulse|heart rate]] as many wearable measure it and the avid self tracker always has a few. | ||
− | |||
− | |||
− | |||
==== Data Cleaning ==== | ==== Data Cleaning ==== | ||
− | Correct, remove or impute outliers (very extreme values) produced by errors but not real events. In the rare case that the data is raw sensor like [[Accelerometry]], aggregate it into something more manageable. Consumer wearables make "steps per 10 minutes" for which open source script is likely available. Finally, compare against other data to remove errors like exercising in the middle of sleep. | + | Check if the device or app produces correct data soon after first use. Correct, remove or impute outliers (very extreme values) produced by errors but not real events. In the rare case that the data is raw sensor like [[Accelerometry]], aggregate it into something more manageable. Consumer wearables make "steps per 10 minutes" for which open source script is likely available. Finally, compare against other data to remove errors like exercising in the middle of sleep. |
== References == | == References == | ||
<references /> | <references /> | ||
− | + | {{Topic Queries}} | |
− | [[Category: | + | [[Category:Topics]] |