1,683
edits
Changes
mLine 24:
Line 24: − +
→Confounders and Artifacts of Procedure
=== Confounders and Artifacts of Procedure ===
=== Confounders and Artifacts of Procedure ===
Any decent skill test will detect when subject is severely sick. For a test to be useful for optimization and experimentation, it must detect more subtle patterns. Statistical tests detect plenty of patterns in my data that are both subtle enough and clearly not generated by the same process as the rest of the data. Unfortunately, those patterns could easily be artifacts of the analysis or of the test taking process. Including all available variables that should NOT correlate with target in a machine learning model and then using resulting residual errors as the final test results should help, but not all variables that cause artifacts are available. Some independent variables will not be useable by the ML model in which case I will plot them against test series with changepoints.
Any decent skill test will detect when subject is severely sick. For a test to be useful for optimization and experimentation, it must detect more subtle patterns. Statistical tests detect plenty of patterns in my data that are both subtle enough and clearly not generated by the same process as the rest of the data.<ref>en.wikipedia.org/wiki/Internal_consistency</ref> Unfortunately, those patterns could easily be artifacts of the analysis or of the test taking process. Including all available variables that should NOT correlate with target in a machine learning model and then using resulting residual errors as the final test results should help, but not all variables that cause artifacts are available. Some independent variables will not be useable by the ML model in which case I will plot them against test series with changepoints.
Some of the artifacts will differ between tests so comparing results from multiple tests may result in a single decent time series strongly dependent on skill. If these solutions are not enough, one of the tests may correlate and transfer to an established validated cognitive test. That would still allow experimentation on and optimization of a skill, but through a cognitive ability.
Some of the artifacts will differ between tests so comparing results from multiple tests may result in a single decent time series strongly dependent on skill. If these solutions are not enough, one of the tests may correlate and transfer to an established validated cognitive test. That would still allow experimentation on and optimization of a skill, but through a cognitive ability.