Editing Flash Cards as Cognitive Test
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
− | + | {{Project Infobox|Self researchers=User:DG|Related tools=Anki,Spaced Repetition|Related topics=Tools for Cognitive Testing|Related projects=Spaced Listening, Spaced Repetition: A Cognitive QS Method for Knowledge Acquisition}} | |
− | Flash cards are cards with question on one side and answers on opposite. They are used for memorization<ref>en.wikipedia.org/wiki/Testing_effect</ref>, making explicit<ref>en.wikipedia.org/wiki/Explicit_memory</ref> (requires effort to remember) declarative<ref>en.wikipedia.org/wiki/Declarative_learning</ref> semantic<ref>en.wikipedia.org/wiki/Semantic_memory</ref> memory though the goal of language learning is to make each memory automatic and therefore implicit | + | Flash cards are cards with question on one side and answers on opposite. They are used for memorization<ref>en.wikipedia.org/wiki/Testing_effect</ref>, making explicit<ref>en.wikipedia.org/wiki/Explicit_memory</ref> (requires effort to remember) declarative<ref>en.wikipedia.org/wiki/Declarative_learning</ref> semantic<ref>en.wikipedia.org/wiki/Semantic_memory</ref> memory though the goal of language learning is to make each memory automatic and therefore implicit.<ref>en.wikipedia.org/wiki/Implicit_memory</ref> Several computer apps automated the process and have recorded a lot of data. I expected electronic flashcard data to be useful as a cognitive test, so I started a project to analyze the data that [[Anki]] records. Turns out the project will teach users about learning and allow them to experiment with and optimize their own learning process. All flashcard apps already optimize their student's learning but do not open the process to the user, except [[Piotr Wozniak|Super Memo 18]]. Resulting visualizations also encourage studying by illustrating success in different way than existing apps, similar to gamification. |
− | + | The project is a work in progress and much of the intended functionality is not yet working. I cannot guarantee that each of the goals will be all that useful to the end user. Project will take about ten thousand lines of code to complete, so I expect to burn out a few times before it is done. No AI or LLM was used in the making of this project. | |
− | |||
− | The project is a work in progress and much of the intended functionality is not yet working. I cannot guarantee that each of the goals will be all that useful to the end user. Project will take about ten thousand lines of code to complete, so I expect to burn out a few times before it is done. No AI or LLM was used in the making of this project. | ||
=== Other Goals === | === Other Goals === | ||
Line 12: | Line 10: | ||
The goal of teaching is almost as easy to guarantee because analyzing the data requires delving into the learning process. This goal may actually be the most impactful. | The goal of teaching is almost as easy to guarantee because analyzing the data requires delving into the learning process. This goal may actually be the most impactful. | ||
− | Project will advise user on how to optimize their studying by comparing what actually happened with what would have happened if they had done something different, | + | Project will advise user on how to optimize their studying by comparing what actually happened with what would have happened if they had done something different, according to a machine learning model. Effects will be illustrated using Partial Dependence Plots and ICE<ref>Visualizing ML Models with LIME · UC Business Analytics R Programming Guide (uc-r.github.io)</ref>. |
To help user conduct experiments, project will compare one part of time series with another or progress on one set of cards with another. User may have to use another tool like [[Open Humans]]. | To help user conduct experiments, project will compare one part of time series with another or progress on one set of cards with another. User may have to use another tool like [[Open Humans]]. | ||
− | |||
− | |||
==== Potential Tests of Emotion ==== | ==== Potential Tests of Emotion ==== | ||
Line 33: | Line 29: | ||
=== Cognitive Health === | === Cognitive Health === | ||
− | Everyone should track their cognitive ability as much as health-conscious people track their heartrate and exercise. | + | Everyone should track their cognitive ability as much as health-conscious people track their heartrate and exercise. IQ tests are supposed to have high 'reliability' and not change much between days for any individual. [[Quantified Mind|Formal cognitive testing]] takes too much time and effect on daily life of the specific thing the cognitive test tests is often questioned. Skill trainers and testers like typing tutors have none of the mentioned problems. However, even if a skill test is useful for optimizing the skill, things like dependence on psychological factors may not make it a good cognitive test. The skill test will have to correlate with validated cognitive tests or obviously important health things or at least transfer to other tests to become validated<ref>[[wikipedia:Validity_(statistics)|en.wikipedia.org/wiki/Validity_(statistics)]]</ref> as a cognitive or health test. The tests are unlikely to be pure in the sense of [[Quantified Mind]]' science page. This makes them better checks for general unhealth but harder to diagnose a specific problem with. |
− | |||
− | |||
− | + | Wozniak of supermemo has found correlation between sleep and flash card skill.<ref>supermemo.guru/wiki/Sleep_and_learning#Studying_sleep_and_learning_with_SuperMemo</ref><ref>supermemo.guru/wiki/Biphasic_life#Biphasic_learning</ref> | |
− | |||
− | |||
− | |||
− | |||
=== Specific cognitive health tests === | === Specific cognitive health tests === | ||
− | Goal of | + | Goal of anki skill is minimize user time required to remember information on a card to a target future date. Decomposition of the general skill<ref>[[wikipedia:Job_analysis|en.wikipedia.org/wiki/Job_analysis]]</ref><ref>Foundations of Fluency: An Exploration: Reading Psychology: Vol 26, No 2 (tandfonline.com) www.tandfonline.com/doi/abs/10.1080/02702710590930519</ref> by looking for sources of time loss results in learning speed, relearning speed, forgetting, ability to remember effectively today, slowness to answer, and distraction. Each corresponds to a potential test and it's validations. New theory of Disuse separates memory into retrieval strength and storage strength or middle and long term memory.<ref>www.learningscientists.org/blog/2016/5/10-1</ref><ref>www.researchgate.net/profile/Robert-Bjork-2/publication/281322665_A_new_theory_of_disuse_and_an_old_theory_of_stimulus_fluctuation/links/58b6f20945851591c5d55e96/A-new-theory-of-disuse-and-an-old-theory-of-stimulus-fluctuation.pdf</ref> |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
==== Recall<ref>supermemo.guru/wiki/Recall</ref>==== | ==== Recall<ref>supermemo.guru/wiki/Recall</ref>==== | ||
− | Remembering on a given day. If long-term then "retention". Wozniak explains it better but I would say that this should be separated in to real forgetting that changes state for several reviews or just momentary lapse. Both are measured separately making two tests. There are also more mild state of card consolidation changes that may happen that are not just forgetting. This may be a third test. | + | Remembering on a given day. If long-term then "retention". Wozniak explains it better but I would say that this should be separated in to real forgetting that changes state for several reviews or just momentary lapse. Both are measured separately making two tests. There are also more mild state of card consolidation changes that may happen that are not just forgetting. This may be a third test. |
− | + | ==== Consolidation of Memory<ref>supermemo.guru/wiki/Memory_consolidation</ref>==== | |
− | + | If on a set day the cards were memorized better or worse as will be seen on their next review. This is like the following two potential identical tests but more generally about change and not about speed? | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
===== Time taken to learn a new word ===== | ===== Time taken to learn a new word ===== | ||
− | + | If considering only the first session, learning new words in Anki is like cued verbal learning like in the CVLT, a cognitive test. Though in the CVLT<ref>en.wikipedia.org/wiki/California_Verbal_Learning_Test</ref> the cue is much weaker. And verbal learning transfers to other abilities: "There is considerable evidence that verbal learning correlates reasonably strongly with performance in a number of important practical tasks. For instance, verbal learning tests have been demonstrated to be highly correlated with prospective remembering in real life, which means remembering to perform a planned action at the appropriate time [29]."<ref>www.quantified-mind.com/science</ref> Sever sleep deprivation harms verbal learning.<ref>pubmed.ncbi.nlm.nih.gov/10688201/</ref> Most of time spent on this task may be on short term memory (10 min) and not mid-long term memory (1 day) so the real test could be percent of successful learnings of new words and not speed. | |
===== Time taken to relearn a forgotten word ===== | ===== Time taken to relearn a forgotten word ===== | ||
Line 83: | Line 61: | ||
Choosing optimal interval (day, month) for next review after seeing answer. Also compared against ML model's recommended amount. | Choosing optimal interval (day, month) for next review after seeing answer. Also compared against ML model's recommended amount. | ||
− | Judgement of learning could be after cue (question) or answer. Delayed, cue only, JOL produced highest correlation with actual recall. If answer is added though, the effect goes way down. Worse still, study decisions are not better than only rereading | + | Judgement of learning could be after cue (question) or answer. Delayed, cue only, JOL produced highest correlation with actual recall. If answer is added though, the effect goes way down. Worse still, study decisions are not better than only rereading.<ref>Kornell, Nate, and Matthew G. Rhodes. "Feedback Reduces the Metacognitive Benefit of Tests." ''Journal of Experimental Psychology: Applied'', vol. 19, no. 1, Mar. 2013, p. 1. </ref> Metacognitive. |
− | |||
− | |||
− | |||
== References == | == References == |