Flash Cards as Cognitive Test

Project Infobox
Self researcher(s)	User:DG
Related tools	Anki, Spaced Repetition
Related topics	Tools for Cognitive Testing
Builds on project(s)	Spaced Listening, Spaced Repetition: A Cognitive QS Method for Knowledge Acquisition
Has inspired	Projects (0)

Flash cards are cards with question on one side and answers on opposite, used for memorization. Several computer apps automated the process and have recorded a lot of data. I expected electronic flashcard data to be useful as a cognitive test, so I started a project to analyze the data that Anki records. Turns out the project will teach users about learning and allow them to experiment with and optimize their own learning process. All flashcard apps already optimize their student's learning but do not open the process to the user. Resulting visualizations also encourage studying by illustrating success in more detail than existing apps, similar to gamification.

The project is a work in progress and much of the intended functionality is not yet working. I cannot guarantee that each of the goals will be all that useful to the end user. Project will take about ten thousand lines of code to complete, so I expect to burn out a few times before it is done. No AI or LLM was used in the making of this project.

Other Goals

Simply adding colorful plots that Anki does not already have will encourage many people to study more. Several variables may potentially correlate with general mood and feelings towards Anki.

The goal of teaching is almost as easy to guarantee because analyzing the data requires delving into the learning process. This goal may actually be the most impactful.

Project will advise user on how to optimize their studying by comparing what actually happened with what would have happened if they had done something different, according to a machine learning model. Effects will be illustrated using Partial Dependence Plots and ICE.

To help user conduct experiments, project will compare one part of time series with another or progress on one set of cards with another. User may have to use another tool like Open Humans.

Potential Tests of Emotion

Tests that should mostly measure user's opinion of Anki itself and possibly general mood. Possibly strongly influenced by outside factors. Useful for optimizing Anki performance but not cognitive testing. For example, expectation of reward (candy) right after session could improve performance by improving mood or opinion of Anki.

• Starting early in the day
• Not skipping days
• Not getting distracted
• Reviewing cards fast
• Reviewing many cards

Confounders and Artifacts of Procedure

Any decent skill test will detect when subject is severely sick. For a test to be useful for optimization and experimentation, it must detect more subtle patterns. Statistical tests detect plenty of patterns in my data that are both subtle enough and clearly not generated by the same process as the rest of the data. Unfortunately, those patterns could easily be artifacts of the analysis or of the test taking process. Including all available variables that should NOT correlate with target in a machine learning model and then using resulting residual errors as the final test results should help, but not all variables that cause artifacts are available. Some independent variables will not be useable by the ML model in which case I will plot them against test series with changepoints.

Some of the artifacts will differ between tests so comparing results from multiple tests may result in a single decent time series strongly dependent on skill. If these solutions are not enough, one of the tests may correlate and transfer to an established validated cognitive test. That would still allow experimentation on and optimization of a skill, but through a cognitive ability.

Cognitive Health

Everyone should track their cognitive ability as much as health-conscious people track their heartrate and exercise. IQ tests are supposed to have high 'reliability' and not change much between days for any individual. Formal cognitive testing takes too much time and effect on daily life of the specific thing the cognitive test tests is often questioned. Skill trainers and testers like typing tutors have none of the mentioned problems. However, even if a skill test is useful for optimizing the skill, things like dependence on psychological factors may not make it a good cognitive test. The skill test will have to correlate with validated cognitive tests or obviously important health things or at least transfer to other tests to become validated as a cognitive or health test. The tests are unlikely to be pure in the sense of Quantified Mind' science page. This makes them better checks for general unhealth but harder to diagnose a specific problem with.

Specific cognitive health tests

Goal of anki skill is minimize user time required to retain information on a card to a target future date. Decomposition of the general skill by looking for sources of time loss results in learning speed, relearning speed, forgetting, ability to remember effectively today, slowness to answer, and distraction. Each corresponds to a potential test and it's validations.

for this or preceding day; forgetfullness on a specific day and change in state (not just forgetting for that one question but really changeing in state.)

Capability to remember borderline cards in this session

Wozniak of supermemo has found correlation between sleep and flash card skill.

Forgetting between sessions

Forgetting as a consequence of something bad happening between test days, like a concussion.

Predicting needs to learn a card

as if the self assement is a matter of how much time will be needed not actual success

Speed of learning a new word

If considering only the first session, learning new words in anki is like verbal learning. And verbal learning, though not connected here to cognitive testing, transfers to other abilities: "There is considerable evidence that verbal learning correlates reasonably strongly with performance in a number of important practical tasks. For instance, verbal learning tests have been demonstrated to be highly correlated with prospective remembering in real life, which means remembering to perform a planned action at the appropriate time [29]."