SenseOS

Project Infobox
Self researcher(s)	Joris Janssen
Related tools	SenseOS
Related topics	Mood and emotion
Builds on project(s)
Has inspired	Projects (0)

Show and Tell Talk Infobox
Featured image
Date	2014/03/17
Event name	Amsterdam Meetup
Slides

This content was automatically imported. See here how to improve it if any information is missing or out outdated.

SenseOS is a Show & Tell talk by Joris Janssen that has been imported from the Quantified Self Show & Tell library.The talk was given on 2014/03/17 and is about Mood and emotion.

Description

A description of this project as introduced by Quantified Self follows:

Joris Janssen is a researcher who’s focused his work on combining sensing algorithms with psychological insights. Currently he’s a researcher and developer at Sense Observation Systems, a Netherlands-based company developing context-aware computing. In this talk, filmed at the Amsterdam QS meetup group, Joris gives a brief explanation of the work they do at SenseOS, then discusses Goalie, an app developed to use psychological theory, active and passive sensing, and a therapy gateway to treat and improve depression.

Video and transcript

https://vimeo.com/91154750

A transcript of this talk is below:

QSAMS10 - Joris Jansen – Sense OS

Thank you very much Mark, and thank you for inviting me. I’m really honored to speak here. Maybe a quick show of hands before Matt sets up the presentation, who has heard of Sense before? Who has heard of self-determination theory before? Not too many people either, that’s very good. So my presentation will be useful I guess. I have an academic background mostly, so I’ve always tried to combine intelligent algorithms with psychological insights. Did a Ph.D. of tracking automatic emotions at TU in Eindhoven. Went on to work at Stanford’s virtually interaction lab. Spent some time at Phillips research, that’s a researcher here. And right now I work at Sense and back as a visiting scholar at Stanford as well. I sort of divided the presentation up into three parts. First I’ll give you a very quick short introduction about what we do at Sense and what we stand for, and then I will talk about self-determination theory and motivation, and then I’ll show you how we applied that theory in one specific project that we did at Sense. I’ll try to go fast but I’m happy to take questions from the audience if there’s something unclear please just interrupt me. It’s a very mixed audience so if I’m not being clear I’ll try to keep you awake until the beer in 25 minutes. So Sense our mission is to turn sensor data into personal context awareness using smart algorithms. We mainly use sensors that everybody is carrying with them on their phone, much like human actually. But instead of what human does with their passive tracking technology, building a generic platform for people, we actually try to use it in all kinds of different applications. Often in procreation with a partner to support a specific group of users, so for instance people with depression or domestic violence or for people with rheumatoid arthritis things like that, and I’ll show you a couple of examples later on. And in everything we do all the users own their own data, and have full control of tier data. It’s unconditional use of privacy. We don’t want to be selling your data, your data is yours. So this is one of the examples that we did. This is a project together with PsyQ, which is a Health Organization and uses an app for people with depression, and this is also the case we’ll take about later on in the presentation. This one the EU Health Award from 2013 from Neilycose supporting people with depression to reactivate using the specific technology. Another example we did together with Apfy and a rheumatoid patient organization. It’s to give people with rheumatoid arthritis more insight in what triggers flare ups of their disease, so pain in their joints and stiffness by tracking in the background in all kinds of things that could trigger this, like the quality of their sleep or their personal weather stream based on location of where they were or how much time they travel, how much time they spend at work or at home; all these kinds of things. The patients can get an insight with these tools, about how these develop overtime and how that actually influences their disease, to better cope with the disease and to better manage their medication better. And then in another app we did together with social care institute in the South of the Netherlands is around domestic violence. It’s actually got a feature that’s show, it tracks emotions and location of partners who are struggling with domestic violence to help them cope better with this by making them aware of what’s going on and also triggering their social network in case it’s necessary. And a very different project, together with Berkeley seismological lab, we were actually using all the sensors that are in your phone to get a better idea of around earthquakes. So this is really using the same technology but for much more research focus so it’s very different tool. So that’s about Sense and to give you an idea of the things we do. We always use these passive tracking technologies and all these different kinds of solutions for different users groups. So now onto the talk about motivation, so I think I can skip this part because I’ve already heard it a couple of times. We tend to look away too much from what we are measuring and why we are measuring, to the measuring itself. And a couple of questions I’ve heard already before the presentations began today is why are we doing all this tracking? There is a multitude of reasons, but one of the biggest ones of course are that you have also seen today already is that it can help you to motivate you, to live healthier. So then the question then becomes, how can we apply self-tracking technology to enhance motivation? I think I can skip this because I think it has been stressed enough, but maybe to just give you an idea, motivation is very important to be healthier, but also for healthier that’s maybe not so obvious like medication adherence. You see a lot of new tools coming out around reminder services or smart pill boxes or things like that because people think that is actually the main problem behind people not being adherent to their medication, which is a huge problem at the moment with 300 billion avoidable cost in the US alone. But research shows that is not actually the main problem behind medication adherence. The main problem and the main reason for non-adherence is 70% of lack of motivation. Once you get patients motivated, they will actually put enough effort into remind themselves, or they will figure out how to take the medication if they don’t know how to take it, all these kinds of things they’re driven by motivation. So motivation is very important. So the next question then is how are we motivating people right now? Of course there’s many ways to motivate people, but I think this is a worrying example that I actually saw popping up this weekend on Twitter. Maybe some of you have seen it as well. On first glance it looks very nice in a new approach to shaping healthy behavior going beyond the one size fits all. But if you start looking at what they are actually doing it’s about a free spa getaway. It’s about a small guaranteed reward to trigger people to eat more healthily. It’s about a free cooking class or a grocery gift card, it’s about more steps than the sales team, or it’s about competing against yourself. It’s about a gift card again. It’s about donating dollars. It’s about a big prize. It’s about a small monthly milestone prizes. It’s about getting your money back about some reason or it’s about more prizes. I don’t know, you probably recognize from my voice I don’t believe this is the way to motivate people and so do a lot of other people also. If we turn to Evgeny Morozov who wrote a very nice book last year, he puts it like this. ‘If you are a narrow-minded economist who believes that we are all utility-maximizing automations, you might be tempted to employ extrinsic motivators everywhere’. Now I hope there’s no economists in the room but I think this captures it quite nicely and also what he mentions here is the difference between extrinsic and intrinsic motivation that underlies a lot of our thinking apparently around what motivation is and how it works. Maybe just a quick example of another situation where intrinsic motivation is actually quite dangerous or not working at all. It’s a story of daycare center that you might have heard before, where daycare centers in the US and across the world often have parents that are late in picking up their kids, which means that the employees have to stay late because they cannot leave the kids and somebody has to keep taking care of them. So the question then became how can we motivate parents to pick up their kids more. So what do you think, let’s put a financial incentive on this. Let’s give them a financial penalty if they come late of course. So what do you think happened? There were more late pick-ups. It’s because parents suddenly didn’t have this social pressure anymore, which actually was a better motivator, but now they can just buy of their guilt with some financial incentive and they were happily willing to do that. The staff gets money out of it and then it can be a good solution. Maybe another solution then came from a recent healthcare report in the US, where a lot of hospitals now have to provide a month guarantee after certain surgeries. So if a patient is re-hospitalized within 30 days of after leaving the hospital after surgery, the hospital actually has to pay for the re-hospitalization and any medical cost that come about there. Now what certain hospitals are thinking there, they’re thinking well why don’t we share that risk partly with the patient – the financial risk. So why don’t we offer some money to patients to be healthy in that 30 day period. Maybe if you don’t smoke or maybe if you keep your blood pressure under control you’ll get some financial incentive for that. And this works for 30 days of course, so for the hospitals this is a good solution, but for the patients this is horrible because after 30 days the extrinsic reward disappears and the behavior goes back to a very unhealthy behavior because they never learn to be actually intrinsically motivated to their behavior. So there’s a risk in extrinsic motivations, that’s the point I was trying to make, so how can we get a better grip on motivation and theories around that. There’s a lot of psychological theories that touch upon it. I really like self-determination theory for a number of reasons. When I’m explaining a theory, please remember there’s a lot of hard evidence behind it, and it might sound a little bit fluffy especially for more technical people. But there’s a lot of stuff and it’s still going on, so there’s a lot of evidence for this theory from the lab where a lot of these were proven and from the real world where more correlations, but it’s a wealth of evidence. And it’s a motivation base frame as opposed to other theories which focus on behavior or intention to change behavior. This is focused around motivation, which is nice if you’re talking about motivation and taking motivation as the core of your theory and it’s very actionable. Actually I’ll just explain the core concepts and I’m pretty sure you’ll be able to apply them to your own situations and a lot of good amount of people already know these principles, and there’s a lot of increasing use that shows this and I know there are a couple of big companies using this like Phillips in designing there products. But I see it popping up in blog posts more and more, so I think this is an interesting theory to be aware of. So what does self-determination propose, it’s that there’s a sort of motivation continuum, which starts at (a) motivation, not being motivated, has a part of extrinsic motivation and end in the ideal position where we all want to be intrinsically motivated to do something. And there’s a lot of steps in between that, so a motivation, no regulation, not being driven to collect certain behavior, but the next step would be external regulation, so this is the first mode of extrinsic motivation and this consists of external pressures. So all the financial rewards, all the punishments that you could get from not doing something, all these kinds of things, external pressures that try to force you to do something. Then there can be something like interjected regulation – I don’t know who makes up these names, but this is about internal pressures so external empathies can put pressure on you but you can also put pressure on yourself through self-esteem, ego or things like that that can pressure you do to something. Identified regulation would be would be one level up, and here you are motivated because of a certain outcome. So maybe you get healthier and that’s why you do something. It doesn’t mean that you inherently like the behaviour that you’re doing, but you are still more likely to do it because of the outcome that generates for you. Then we get into integrated regulation and this one is probably the least initiative or the hardest to explain to a bunch of western technologists, but this is to do with the coherence with yourself, and the example that is often given is following a religion or going to church. This is also something that you might not inherently enjoy the activity itself but it gives you a lot of pleasure through your personal growth that it gives you. And then finally what is called intrinsic regulation, so this is when you do an activity purely out of interest and enjoyment for the epiphany itself. And of course this is the biggest motivator to do something. Now is it bad if you are doing something and your motivation is at a lower level? Probably not and we cannot change all your behaviors to an intrinsic motivation level, but it’s much more sustainable and you will probably be able to keep up with the behaviour for much longer, so we should still try. Just to give an example of course, it’s not about the behavior itself but it’s about why you conduct in that behavior. So you can exercise to lose weight, to look good or to prevent heart disease. And this would mean a lot of self-determination, it’s a lack of choice and this internal pressure in this case or you are just doing it for the outcome. Whereas you could also exercise for enjoyment, for feeling competent, for social affiliation that you lead to exercising and this would lead to high self-determination. So now the question becomes, how can we increase the self-determination of people, and what the theory proposes is out of all the things that we can do there’s actually three fundamental psychological things that we need to stimulate to make people more self-determined and more motivated. So the first one is competence; if you think you cannot do it, you will not do it. and if you believe in yourself you are much more likely to pursue a certain activity, of course yoga is a perfect example of this. You have to start believing in what you can do to be able to be more motivated to do it. The second one is autonomy. So nobody wants to be forced or pressured into doing something and we always have to have the feeling that we have our own choice that we can make. We don’t want to be cohered, we want to have freedom to choose in what we do, and as long as you’re given room to choose for yourself, to think for yourself, to explore options, you’re much more likely to be motivated by the choice that you actually make. Finally is relatedness. The is Wilson from the movie Castaway and maybe some of you remember this about a guy who gets stuck on an island for I don’t know, 15 years or so. And as humans we are all social animals and we actually have a deep need for social interaction, for deep social connections or what they call it relatedness, so also this castaway guy he made this imaginary friend, Wilson a face on a football. But we are always looking for social interaction and for deep social connection. So if there is some way to stimulate that that will tremendously help with this to be motivated. So how can we do that? Just to give you some ideas stimulating competence, present clear, neutral information about behavior outcomes. Set appropriate and realistic goals and expectations and don’t set your goals too high. Personalize the goal on your level and what the CEO of Human said earlier for those Americans who only move five minutes a day or not even and 30 minutes is way too high, so they will stop doing the behavior immediately. Provide positive feedback and positive feedback only, no negative feedback it will decrease motivation and support self-advocacy. This is also a term that is also associated with competence that you might have heard before, so it’s to believe in what you can do yourself. Supporting autonomy, I already said it a little bit, but determine what is important for the user and the user has to decide what is important for them, try to translate their goals into something actionable that you can do with your tool for instance but they have to set their own goals, Explore options, avoid cohesion, and minimize pressure and let the user make the decision in what and how to change. And build a relationship for relatedness, it’s pretty obvious. Express empathy and express understanding. This might be hard for a computer to do but I will come back to this later and give some examples of how you can do that. Avoid judgement and blame and show genuine interest. Now this might sound all very obvious to some of you but there’s a lot of tools that actually don’t do this. It maybe sound very easy to implement this, but this is so often forgotten that I think we should try to remember it better. So how can this self-tracking technology or this passive tracking to enhance intrinsic motivation I’ll show that using the example of this mHealth platform for depression. So here the problem is patients with depression often at the beginning of their treatment are in this very passive mode; they don’t get out of bed, they stay on the couch for the entire day, they often skip meals, they eat very unhealthy, they go to bed too late, they sleep into the day, they don’t have any social contact anymore etc. they are stuck in that situation. And of course they end up in therapy; the therapist will try to do a lot to get them out of this situation but some more tooling during the day, with the idea will greatly help them. So we got together with therapists from PsyQ to see if we can make a platform that could support the interaction between the therapist and the patient. Try to get the patient more motivated to be a little bit more active during the day. This is what the app looks like for the patient. So the app tracks a number of this, sleep, exercise, movement, locations, emotions, and food intake. This is actually active tracking because we cannot measure what you eat at the moment. And together with the therapist the patient’s goal will be simple low level goals at the beginning and maybe only activating one of those goals or two of those goals and slowly building it up when things are going better. And the patient gets positive feedback during the day when things go well. What is important there is that this is real time process so that we can also give the feedback right next to when the behavior occurs. This is very important to make the link much more explicit. So normally, the patients would maybe track something for themselves, and come back after a week or two weeks to the therapist and say well done. But now they hear it exactly when the behavior occurs and it makes it much more motivating for them to keep doing it. So right time feedback. And the therapist also has insight into all of these behaviors so there’s a weapon to face for the therapist who can sort of look over the shoulder of the patient and see how things are going and adjust the therapy accordingly based on the things that are going well or not going well and put more emphasis on that and that was the idea. I think this is already clear from the presentations before, but we use smart phone, a lot of sensors in there and track a lot of different things so just to give you some examples, this is my battery level for a couple of days, and that already gives you some insights about my behavior, how I live. This is my location trace; I live in Eindhoven and work in Rotterdam and should not recommend that to anyone. Here is my ambient noise pattern from the microphone so we don’t actually record audio but we track the number of decibels, and it sort of gives you an idea in the amount of noise in the environment. And here you see three axis on the barometer data, and now if you look on the x-axis, you see five consecutive days and you probably already see a nice pattern in there. Right so, what you see is during the night my phone is lying still and the environment is relatively quite, whereas during the day my phone is moving and there’s much more noise. Combine this with the ambient light sensor you also have in your phone, you can get quite an accurate sleep tracker. So this kind of technology we try to use to build these applications on top of. This is the dashboard for the therapist that I already mentioned. So the therapist configures together with the patient. The goals are visit them every week, every two weeks to see how things are going. The patient sees these kinds of interfaces, showing their daily progress or weekly progress around sleep or around exercise and see if they managed to get their goal for that day, what their goal is, and how active they have been and get some advice below there on why this is important. And then there’s the interface for the therapist who gets a long term overview of how things are going. So that’s in a nutshell. So how does this support self-determination? Somethings maybe a little bit obvious but I’ll got through them. So positive reinforcement, we don’t give any negative feedback; it’s always positive. This is what I already mentioned and this supports competence as well. We get the feedback at the right time together with when the behavior occurs, so it’s much more likely to have an impact. How does this support relatedness? It might be a little bit less obvious. But this is interesting and this is based on the media equation work of Reese and Ness, which basically says that the way we interact with computers is the way we also treat other humans. This is also from 1997 and there’s a lot of research that shows that we are much more likely to judge a computer – just let me give an example, otherwise it’s too crazy to believe. So they did a study where they have people doing a task on a computer and then later on evaluate that computer through a questionnaire. Half of the people did the evaluation on the computer that they were evaluating. The other half of the people did the evaluation on a different computer that was explicitly told that the computer was no connection to the original computer. What do you think happened? Well, the people who did the evaluation on the same computer were more positive out of a sort of a nice social gesture towards that computer. What is interesting about this as well is that these were Stanford computer science students, and afterwards of course they were asked, did you think your evaluation was influenced by the fact that you did this on the same computer, of course not and they laughed the researcher away so to speak. They found the whole idea ridiculous but still there was very significant difference in their evaluations. So the social rules that we use in our day to day life to interact with other people also apply to computers. It’s also the same response that we got from patients that actually used the app for a while. After the piolet period we wanted to take it away from them and say well done and nice that you used it for a few months, and very nice that you were so enthusiastic about it, and now the piolet has stopped. And they didn’t want to stop using it. They said it is my body, it’s my friend. He really understands me, right, and these kinds of things in the feedback that we got. So this sort of gives them a social connection to the app. The other point of course is that through this app the patients share a lot of personal information with the therapist and this builds up a relationship as well, and makes them trust the therapist more and makes them take the advice from the therapist much more seriously. This is a sort of a basic psychological process that’s not so eciting but it still works. Then around supporting autonomy, because of course if you look at self-determination that’s also the weakest point of this app, but we stills et goals with a target that you have to reach and of course this is safeguarded by the fact that there is a therapist in the loop, but there is other things that we can do to support autonomy with this kind of data which we are not doing yet but we are planning on doing. So once you can start tracking all these things and combine it with how people are feeling, let’s assume that only on the y-axis here you have your wellbeing level. This forms this energy landscape across all kinds of different dimensions that may influence your life. So if the social situation that you are in, or the work situation that you are in, once we start to understand that we can also know and explore this energy landscape. So once you go through life you will get a better idea of what works for you well, and when you feel well and when you don’t feel well. This will help you reach the optimum for you, which of course is very different from other people. Interestingly of course maybe you are in a group of friends which is so-so, which will be very hard of course to make it very different to step to a different group of friends because you have to go down in wellbeing first to get to this new situation. This is the way these therapists at PsyQ are thinking about this because this will really help them to better coach there patients. Now I think I will skip this, but I think I give a nice idea about self-determination theory and what it can mean. If you want to try out some of the tracking technology for yourself we have a common-sense dashboard. It’s a sort of a free tool and an app that you can install on your phone if you don’t want to buy expensive trackers and want to get started with Quantified Self. You can have a look at it and it’s a private app, but I’ve made sure there is enough tokens available tonight if people want to try it. www.commonsense-dashboard.com

that’s what I wanted to talk to you today so thank you very much.

About the presenter

Joris Janssen gave this talk.