Changes

{{Project Infobox|Self researchers=User:DG|Related tools=Anki,Spaced Repetition|Related topics=Tools for Cognitive Testing|Related projects=Of Trivial Value? Lessons From Using SuperMemo, Spaced Repetition: A Cognitive QS Method for Knowledge Acquisition, Knowledge Tracking, Memory and Learning}}  

Main place for news of this project is on a [https://forums.ankiweb.net/t/really-thorough-statistical-analysis-of-your-data/36355 thread in Anki forum] because most of the audience would be there.{{Project Infobox|Self researchers=User:DG|Related tools=Anki,Spaced Repetition|Related topics=Tools for Cognitive Testing|Related projects=Of Trivial Value? Lessons From Using SuperMemo, Spaced Repetition: A Cognitive QS Method for Knowledge Acquisition, Knowledge Tracking, Memory and Learning}}  

Flash cards are cards with question on one side and answers on opposite. They are used for memorization<ref>en.wikipedia.org/wiki/Testing_effect</ref>, making explicit<ref>en.wikipedia.org/wiki/Explicit_memory</ref> (requires effort to remember) declarative<ref>en.wikipedia.org/wiki/Declarative_learning</ref> semantic<ref>en.wikipedia.org/wiki/Semantic_memory</ref> memory though the goal of language learning is to make each memory automatic and therefore implicit,<ref>en.wikipedia.org/wiki/Implicit_memory</ref> system one.<ref>bokcenter.harvard.edu/how-memory-works</ref> Several computer apps automated the process and have recorded a lot of data. I started this project to analyze the data that [[Anki]] records because I expected electronic flashcard data to be useful as a [[Tools for Cognitive Testing|continuous cognitive test]]. If this works, the resulting information would be useful for Citizen Science and Health Tracking. Academic papers may be successfully replicated here even though neither complicated multi-step analysis (though common some places<ref>www.kaggle.com/competitions?hostSegmentIdFilter=2&searchQuery=university</ref><ref>paperswithcode.com/dataset/shhs</ref>) nor single subject longitudinal observational studies like this project are common on google scholar.   

Flash cards are cards with question on one side and answers on opposite. They are used for memorization<ref>en.wikipedia.org/wiki/Testing_effect</ref>, making explicit<ref>en.wikipedia.org/wiki/Explicit_memory</ref> (requires effort to remember) declarative<ref>en.wikipedia.org/wiki/Declarative_learning</ref> semantic<ref>en.wikipedia.org/wiki/Semantic_memory</ref> memory though the goal of language learning is to make each memory automatic and therefore implicit,<ref>en.wikipedia.org/wiki/Implicit_memory</ref> system one.<ref>bokcenter.harvard.edu/how-memory-works</ref> They do this through the principal of spaced repetition.<ref><nowiki>https://www.gwern.net/Spaced-repetition</nowiki></ref> Several computer apps automated the process and have recorded a lot of data. Inspired by [[Piotr Wozniak|Wozniak's]] and Gwern's<ref>gwern.net/treadmill#treadmill-effect-on-spaced-repetition-performance-randomized-experiment</ref> experiments, I started this project to analyze the data that [[Anki]] records because I expected electronic flashcard data to be useful as a [[Tools for Cognitive Testing|continuous cognitive test]]. If this works, the resulting information would be useful for Citizen Science and Health Tracking. Academic papers may be successfully replicated here even though neither complicated multi-step analysis (though common some places<ref>www.kaggle.com/competitions?hostSegmentIdFilter=2&searchQuery=university</ref><ref>paperswithcode.com/dataset/shhs</ref>) nor single subject longitudinal observational studies like this project are common on google scholar.   

Turns out the project will teach users about learning and allow them to experiment with and optimize their own learning process. All flashcard apps already optimize their student's learning but do not open the process to the user, except [[Piotr Wozniak|Super Memo 18]]. (this project already has different features) Resulting visualizations encourage studying by illustrating success in different way than existing apps, similar to gamification.   

Turns out the project will teach users about learning and allow them to experiment with and optimize their own learning process. All flashcard apps already optimize their student's learning but do not open the process to the user, except [[Piotr Wozniak|Super Memo 18]]. (this project already has different features) Resulting visualizations encourage studying by illustrating success in different way than existing apps, similar to gamification.   

Cortex, Volume 39, Issues 4–5, 2003,  

Cortex, Volume 39, Issues 4–5, 2003,  

doi.org/10.1016/S0010-9452(08)70878-5</ref>         

academiccommons.columbia.edu/doi/10.7916/9p9w-tc61/download</ref>         

=== Specific cognitive health tests ===

=== Specific cognitive health tests ===

Goal of Anik skill is to minimize user time required to remember information on a card to a target future date. Decomposition of the general skill<ref>[[wikipedia:Job_analysis|en.wikipedia.org/wiki/Job_analysis]]</ref><ref>Foundations of Fluency: An Exploration: Reading Psychology: Vol 26, No 2 (tandfonline.com) www.tandfonline.com/doi/abs/10.1080/02702710590930519</ref> by looking for sources of time loss results in learning speed, relearning speed, forgetting, memory consolidation, ability to remember effectively today, slowness to answer, and distraction.<ref>compmem.princeton.edu/wp/wp-content/uploads/2020/04/replay-based-consolidation-governs-enduring-memory-storage.pdf</ref> Each corresponds to a potential test and it's validations. Consider "test-retest" reliability<ref>www.tandfonline.com/doi/abs/10.1080/13854046.2017.1310300</ref> and applicability to longitudinal studies. New theory of Disuse separates memory into retrieval strength and storage strength or middle and long term memory.<ref>www.learningscientists.org/blog/2016/5/10-1</ref><ref>www.researchgate.net/profile/Robert-Bjork-2/publication/281322665_A_new_theory_of_disuse_and_an_old_theory_of_stimulus_fluctuation/links/58b6f20945851591c5d55e96/A-new-theory-of-disuse-and-an-old-theory-of-stimulus-fluctuation.pdf</ref>

Goal of Anik skill is to minimize user time required to remember information on a card to a target future date. Decomposition of the general skill<ref>[[wikipedia:Job_analysis|en.wikipedia.org/wiki/Job_analysis]]</ref><ref>Foundations of Fluency: An Exploration: Reading Psychology: Vol 26, No 2 (tandfonline.com) www.tandfonline.com/doi/abs/10.1080/02702710590930519</ref> by looking for sources of time loss results in learning speed, relearning speed, forgetting, memory consolidation, ability to remember effectively today, slowness to answer, and distraction.<ref>compmem.princeton.edu/wp/wp-content/uploads/2020/04/replay-based-consolidation-governs-enduring-memory-storage.pdf</ref> Each corresponds to a potential test and it's validations. Consider "test-retest" reliability<ref>www.tandfonline.com/doi/abs/10.1080/13854046.2017.1310300</ref> and applicability to longitudinal studies. New theory of Disuse separates memory into retrieval strength and storage strength or middle and long term memory.<ref>www.learningscientists.org/blog/2016/5/10-1</ref><ref>www.researchgate.net/profile/Robert-Bjork-2/publication/281322665_A_new_theory_of_disuse_and_an_old_theory_of_stimulus_fluctuation/links/58b6f20945851591c5d55e96/A-new-theory-of-disuse-and-an-old-theory-of-stimulus-fluctuation.pdf</ref>

==== Consolidation of Memory<ref>supermemo.guru/wiki/Memory_consolidation</ref><ref>www.ncbi.nlm.nih.gov/mesh/2009921</ref>====

==== Consolidation of Memory<ref>supermemo.guru/wiki/Memory_consolidation</ref><ref>www.ncbi.nlm.nih.gov/mesh/2009921</ref>====

If on a set day the cards were memorized better or worse as will be seen on their next review. This is like the following two potential identical tests but more generally about change and not about speed?     

If on a set day the cards were memorized better or worse as will be seen on their next review. This is like the following two potential identical tests but more generally about change and not about speed?     

==== Recall<ref>supermemo.guru/wiki/Recall</ref>====

==== Recall<ref>supermemo.guru/wiki/Recall</ref>====

Neurology Jan 2003, 60 (1) 82-86; DOI: 10.1212/WNL.60.1.82 </ref>

Neurology Jan 2003, 60 (1) 82-86; DOI: 10.1212/WNL.60.1.82 </ref>

===== Time taken to learn a new word =====

===== Time taken to learn a new word =====

Short-term memory, from seconds to hours.<ref>Memory, Short-Term - MeSH - NCBI (nih.gov) www.ncbi.nlm.nih.gov/mesh/68008570</ref> If considering only the first session, learning new words in Anki is like cued verbal learning like in the CVLT, a cognitive test. Though in the CVLT<ref>en.wikipedia.org/wiki/California_Verbal_Learning_Test</ref> the cue is much weaker. And verbal learning transfers to other abilities: "There is considerable evidence that verbal learning correlates reasonably strongly with performance in a number of important practical tasks. For instance, verbal learning tests have been demonstrated to be highly correlated with prospective remembering in real life, which means remembering to perform a planned action at the appropriate time [29]."<ref>www.quantified-mind.com/science</ref> Sever sleep deprivation harms verbal learning.<ref>pubmed.ncbi.nlm.nih.gov/10688201/</ref> Most of time spent on this task may be on short term memory (10 min) and not mid-long term memory (1 day) so the real test could be percent of successful learnings of new words and not speed.  

Short-term memory, from seconds to hours.<ref>Memory, Short-Term - MeSH - NCBI (nih.gov) www.ncbi.nlm.nih.gov/mesh/68008570</ref> If considering only the first session, learning new words in Anki is like cued verbal learning like in the CVLT, a cognitive test. Though in the CVLT<ref>en.wikipedia.org/wiki/California_Verbal_Learning_Test</ref> the cue is much weaker. And verbal learning transfers to other abilities: "There is considerable evidence that verbal learning correlates reasonably strongly with performance in a number of important practical tasks. For instance, verbal learning tests have been demonstrated to be highly correlated with prospective remembering in real life, which means remembering to perform a planned action at the appropriate time [29]."<ref>www.quantified-mind.com/science</ref> Sever sleep deprivation harms verbal learning.<ref>pubmed.ncbi.nlm.nih.gov/10688201/</ref> Most of time spent on this task may be on short term memory (10 min) and not mid-long term memory (1 day) so the real test could be percent of successful learnings of new words and not speed. Age and memory complaints correlate with poorer cued recall at 55min delay and the test may identify subtle impairments.<ref>pubmed.ncbi.nlm.nih.gov/35786222/</ref>

===== Time taken to relearn a forgotten word =====     

===== Time taken to relearn a forgotten word =====     

Judgement of learning could be after cue (question) or answer. Delayed, cue only, JOL produced highest correlation with actual recall. If answer is added though, the effect goes way down. Worse still, study decisions are not better than only rereading. This seems to be because users do not realize they learned because of last test.<ref>Kornell, Nate, and Matthew G. Rhodes. "Feedback Reduces the Metacognitive Benefit of Tests." ''Journal of Experimental Psychology: Applied'', vol. 19, no. 1, Mar. 2013, p. 1. </ref> Metacognitive. Making JOLs also increases memory of items, maybe.   

Judgement of learning could be after cue (question) or answer. Delayed, cue only, JOL produced highest correlation with actual recall. If answer is added though, the effect goes way down. Worse still, study decisions are not better than only rereading. This seems to be because users do not realize they learned because of last test.<ref>Kornell, Nate, and Matthew G. Rhodes. "Feedback Reduces the Metacognitive Benefit of Tests." ''Journal of Experimental Psychology: Applied'', vol. 19, no. 1, Mar. 2013, p. 1. </ref> Metacognitive. Making JOLs also increases memory of items, maybe.   

== References ==  

== References ==