Marvellous Capability: Mental health and ‘people dependent technology’.

Internet and Everyone (Brain Pickings)

This is a lovely page, from a book called ‘The Internet and Everyone’ by John Chris Jones, which I came across on the Brain Pickings site a few years ago. It rang true to me then, and even moreso now, that the philosophy the author talks about is one we really should try to embrace when designing mental health technologies. In this section on “people dependent technology” he makes a plea that:

“We design everything on the assumption that people are not heartless or stupid but marvellously capable, given the chance”

I love that phrase, ‘marvellously capable’. Certainly the first generation of mental health technologies I’ve come across, the online packages of digital CBT for example, seem designed as if the people using them are heartless (in the sense of unemotional or willing to drudge through drudgery with no personal sense of frustration or need – quite the opposite of course to how people really are, and certainly people experiencing a mental health problem) and stupid (in the sense of needing the most obvious things explained ad tedium but given no opportunity to be flexible, creative or to personalise the system the way they see fit.)

He also writes:

“The main obstacles to this at present are not so much the machines or technical processes, but the presence of our other selves…enforcing rules of behaviour and design which assume that ‘users know nothing and producers know all.”

The last bit is becoming something of a mantra of mine – I’ve spoken before on this blog about the need to ‘follow the user‘ and how technology provides us with great examples of ‘bricolage’, of users blending and mending tech into their own lives in ways that suit them, and ways that the original producers and designers should learn from.

But the first bit really strikes me as well – the idea that is ‘our other selves’ that we insert into software that really prevents people from doing what is best. Those online CBT packages for example are designed almost as if to deliver therapy the way a therapist would – you have a 1 hour session, you sit and listen and are expected to learn, and you get given homework that you report on next time. It’s absolutely daft of course that a technology, which could be made so flexible and integrated so much more elegantly into everyday life, is still following a set 1-hour-a-week routine as if the patient is turning up at their therapist’s office for their regular slot. But I think it goes deeper than that. The fact that the programme still assumes the role of the patient is to receive information and respond to instruction – this doesn’t seem like a design that recognises people as “marvellously capable, given the chance.”

I think this is a great shame. I think it would be very sad if we abandon the idea of mental health technology because this first generation got in the way of people, rather than supporting them to do better. I think this is especially true given we have such a clear path laid out for making the next generation better – follow the user, and embrace the idea of “people-dependent technology”. The technology depends, not on us as the designers, to be clever and assume we know what to produce, but instead on the people who are using it.

Posted in Mental Health, Technology | Tagged , , | Leave a comment

Open Academia: Embracing interaction over transmission

Victoria Betton made some excellent observations earlier this week on the challenges of straddling the open world of social networking with the formal and closed traditions of academia. One point, drawn from David Price’s book on open learning and sharing, was the observation that most schools and workplaces adopt ’transmissive teaching’, a top down, linear approach to depositing facts into peoples brains – giving the lecture from the front of the room without responding to the audience, or even inviting any response. By contrast, social media and open working encourage and require dialogue and interaction. It’s a two way process, and both parties give and receive (with often the really neat stuff happening somewhere in the middle).

That word ’transmission’ really stuck out for me – the one-way only route of ‘I send something, you receive it’. I think ’transmission’ sums up how quite a few Universities currently use social media – they transmit their achievements, in press releases or latest papers, and interaction with anyone except their own or other research groups is rare. I think this is partly a resource and training issue. I think genuinely interacting on twitter, particularly if tasked with representing an official organisation (who is most likely your employer), can be complex and time consuming. Most people tasked with manning the ‘official’ twitter feed aren’t given any formal support or training on how to do this, and I suspect most are simply given the task of transmitting in any case. I think this is how many universities view this kind of public engagement – we announce stuff, you pay attention. We tell you stuff, you listen. Of course, this isn’t actually how twitter works, and I don’t think it should be how public engagement works either.

There's a somewhat catty attitude on display in academia's approach to hearing service user voices.

There’s a somewhat catty attitude on display in academia’s approach to hearing service user voices.

This transmission approach I think is also evident though in how some academics approach PPI (patient and public involvement), where PPI is about them telling service users something, and opportunities for real two-way interaction and collaboration seem minimal. This attitude doesn’t recognise that it might be us who needs to do the learning.

Don’t give up on us just yet though. There is a move toward openness in academia, with the #openaccess movement and the #alltrials initiative being prominent examples. It strikes me though that both of these relate to our findings being made more open, and calls for the process of what we do to be more open are less prominent. For me, this is where PPI sits – it’s about opening up that research process, what we do, how and why. It’s about sharing those processes with service users, inviting them to join in and not just to listen. It’s about being transparent about our decisions and priorities, and letting service users themselves teach us about what their priorities might be. This has to be an interaction, and ideally a collaboration.

The irony here is that of all the disciplines that could be open, academia and scientists in particular should be leading the way. One of the core principles of science, and to me one of its most appealing, is transparency. Science is the rejection of authority – you do not tell people something is true, you demonstrate it, you show your working, you open it up to debate. Looked at in that way, those who call for greater interaction with users are the ‘pure scientists’ in comparison to those who think we should keep our doors closed.

Posted in Thinking about research | Tagged , , | Leave a comment

Open eyes, open mind? How open access could support recognition of patient views

There’s a great post over at the LSE Impact Blog today by Jonathan Gray which makes both a practical and a philosophical argument for open access publishing in academia. Open access refers to making research papers actually publicly accessible, as currently most journals we publish in require hefty subscription fees to read.  Universities tend to be their biggest customers, with the result that most research is published only to be read by other researchers. This article makes a lovely argument about the responsibility of we ‘scholars’ to share knowledge and to encourage discussion and debate, which locking findings behind closed web links runs counter to.

For me, the debate about open access is a debate not just about the availability of research findings but about the responsibility and duty to provide those findings to the public who so often pay for them. However, open access discussions tend to be focused on – or perhaps at – journals and their publishers, and researchers and their universities. The majority of researchers will be working at institutions which have subscriptions to the journals, so this isn’t about researchers themselves struggling to get the papers (though this does happen – #icanhazpdf on twitter is now used by researchers to ask academics at other universities whether they can get a paper that the researcher’s own university doesn’t subscribe to.) But I did notice that this meant the argument about “open access for all” is more one of principle then (1), rather than making an argument with, for example, patients themselves about their need or desire to access those findings.

I can haz journal articles? Pleazzze?  (image via

I can haz read journal articles? Pleazzze?
(image via

A while ago I began to wonder if this really was even an issue for people outside academia though – do they really ever try and read these papers? I asked as much on twitter, and various patients/service users did reply and say this was an indeed is an issue for them, that they had wanted to read the results of a study about their health issue but found themselves locked out. Personally I think this supports the arguments above, that if we are ostensibly doing research for patients, then locking the findings out of their view is kind of crappy. What I found really enlightening though was the discussion of how these paywalls made them feel.

As is often the case with issues when you ask patients or service users directly, you find out about an angle you hadn’t considered. It hadn’t occurred to me that closed access publishing could be seen as yet another symptom of a research culture that doesn’t really rate the end users of it’s research. It made me realise that perhaps another benefit of open access would be to encourage (or perhaps force…) researchers to engage with “their” public, the people they did the research ‘for’. Perhaps if we let patients read about research, we’d be able to engage in educated discussions with them about it, and we might start to realise what they have to contribute. As long as the findings are locked away, for our eyes only, we can carry on telling ourselves that only we understand it, and hey, it’s all for their own good anyway, right?


(1) There is a more economic argument as well – given that Universities are almost constantly complaining about lack of funding, perhaps we should look at which industries we line the pockets of:

PS. A huge thank you to those tweeps above who contributed to the discussion and helped me make these realisations!



Posted in Academic Stuff | Tagged , , , | 1 Comment

Post Doc Or Not? 4 questions to help you decide.

You’re coming to the end of your PhD. You might even have handed the b*stard in. Maybe you’ve even survived your viva. You drop to the floor, let out a huge sigh of relief, wait for the trumpets to sound – but then:

“So, what are your post doc plans?”

Aaaand *deflate*.

This celebration feels less... celebratory. It feels more... impending doom.

This celebration feels less… celebratory. It feels more… impending doom.

(I’m sure some people doing a PhD would debate the above. They’d point out that they’ve been asked about their ‘plans for post doc’ since Year 1. Maybe even since the PhD interview itself.)

We held a sort of informal forum meeting yesterday, where a panel of research fellows (including me) tried to speak candidly to PhD students and junior researchers about the actual process of ‘doing’ a Post Doc – finding jobs, finding funding, looking for fellowships and so on. I hope people found it useful. I think it can be helpful though to take a step even further back, and to ask very fundamentally “Do I want to be a post doc?” The points below are the questions I think you should ask yourself if you’re in that position.

Now, to be clear: the very very first question you should ask is “Do I want to stay in academia?” Don’t take it as a given that you should.  You could be conned into thinking that you’re now such a specialist that there aren’t any other jobs you could get (check out the stories on #altac on twitter to see there are other careers beyond a PhD.) You may also fall pray to the idea that as you’ve thrown the last 3 – at least –  years into academia, you should stick it out and keep going. I feel like this is the lifeyears equivalent of throwing good money after bad, if you haven’t particularly enjoyed those 3 years at all.

So, first of all – can you see yourself working in a University? If you think the answer to this is “yes!” or “yeah, maybe” that’s when the questions below could come in handy, to think through what that means in more detail. Fundamentally though, you shouldn’t try to talk yourself into it if it’s not what you want, and don’t let anyone else talk you into it either.

The Questions:

1. Can you tolerate being on short term contracts?

In practical terms, as a post doc you are not employed by a University as such – you are employed on a specific grant, with a specific timescale. This is typically 1-2 years, but can be as low as 6 months and if you’re really lucky go up to about 4 or 5 years, say if you’re employed on a large programme grant. But at the end of this, you’re out of contract, and you’re on your own. In good departments, your colleagues or supervisors will support you to get another grant, so you can keep working. But grants are never a given, and most are rejected. In this sense a post doc is more like a freelancer – you’re paid to work on specific projects and there’s no safety net in-between. On your current job you’re almost always already busy working on the next grant, with an eye on your contract running out. Is this something you want to deal with?

I want to make very clear here that ‘tolerance’ is not some kind of test of strength (which is how it gets portrayed sometimes – “Can you stick it, sunshine? Not having a guaranteed job in 6 months? It’ll be the making of you!”). It’s also not an individual attribute – if you have a family, or caring responsibilities, then this kind of insecurity may just be infeasible. But you need to have realistically considered whether you’re willing to put up with the  job insecurity that goes hand in hand with early career research work.

Obviously the financial insecurity  part of this is a big concern. But it’s not just an issue of money. You could have a lovely patron who’ll keep you going between contracts and still be frustrated that you can’t establish yourself in a particular University or research area because you’re at the mercy of whatever jobs become available in whatever place. Other people I’ve known have quite enjoyed this aspect in that they actively move around to different locations each time, but for those who want to put down roots it can be problematic.  Personally, I find the insecurity frustrating but tolerable. I’m relatively lucky in that I’ve only fallen through the gaps between short term contracts once and had a couple of months without pay. I’ve only had one major move (from Nottingham to Manchester) and it worked out very well for me.  It’s definitely scary though how quickly savings can dry up though once you’re off the payroll.

2. Do you like ‘flexible working hours’?

Now, contrary to what people with fixed working hours assume, this doesn’t mean more time off/’working’ from home whilst actually playing computer games/starting late and leaving early etc. Flexible working in reality means you’re expected to do as much work as necessary to get the job done. This can mean giving up weekends if a grant is due, or having to drop everything if you get landed with a particular task that needs doing RIGHT NOW.  Managing your workload independently can mean you over burden yourself and do more than you should, and there’s no such thing as overtime in academia.   You’re also expected to make your own time to do additional (required) activities such as teaching. I’m expected to do a certain amount of teaching per semester, but I don’t get to recoup this in terms of additional days for research at the end of my contract or additional money.

On the positive side,  I like managing my own workload and deciding (for the most part) what to do and when. I know people in fixed jobs have to do things like take a day off to go to the dentist which we would never do – we just make up that work later. Outside of teaching and department events, you can often work from home. The trick with that is knowing when to turn off and ‘go home’ psychologically.

This is the less tangible but perhaps more pernicious impact of ‘flexible working’ – it can be hard to put work aside, both mentally and physically (email has made it much easier to cheat on your weekend and spend time doing work). I don’t mind this aspect too much, when it involves thinking more about the bits of science or research that I find really interesting. In that sense I’m exceptionally fortunate to do something for a living that I’d do for a hobby anyway. The difficulty comes when you get stuck worrying about issues that you can’t control or feeling guilty about not doing work all the time. These are more personal, and I actually think a lot – if not most – academics struggle with them to an extent. I think if you’re someone who likes very clear borders for being ‘at work’, if you like regular hours and timetables, or if you struggle with motivating yourself to deal with deadlines without supervision, then a post doc or fellowship is definitely not for you.

If you like the idea of being your own boss when it comes to your timetable, and don’t mind changes in pace (I actually think I work more naturally with periods of intense action followed by slower refuelling periods – I know in temp jobs I used to find having the same hours/activities every day very dreary) then a research career could suit. If you’re the latter, but worried about getting overburdened, then I’d reassure you that this is a worry for a lot of us, and so there are sources of support, both offline and on, to help you remember to balance your priorities. You will however just have to cope with the fact that everybody and their dog thinks you work in your lounge pants for a few months a year and then get the really long summer holiday with the students.

This working from home business is more tiring than you might imagine.

This working from home business is more tiring than you might imagine.

3. Do you have an itchy brain?

I talked about this in my ‘qualities for a PhD student’ post and the same applies here, but ramped up another level. This is because in a post doc, and especially in a fellowship, you’re going to be expected to producing ideas and identifying new avenues of research. The itchy brain moves from “hmm, I really want to know more about that to…” to “I need to know something about this that no-one else can tell me”, because it hasn’t been done or hasn’t been considered – and it’s this gap that you then try to fill.

This is definitely more of an issue in fellowships (independent programmes of work/training) compared to working as a jobbing post doc on other studies. I actually think I worked longer hours on my early post doc positions, in terms of delivering studies and doing the leg work of trials, but these are quite task based and you can get quite efficient in having a routine. I think at fellowship level it becomes more a matter of intellectual energy (I know that makes me sound horribly pretentious, but I genuinely can’t think of a better way to put it!) It’s not “Do X before 5pm”, it’s “Think of/create/write X before 5pm”. For me, this is brilliant, and I vastly prefer it to the more task based work I used to do. For this reason, I actually consider that work ‘harder’ than the work I do now – because for me it was less interesting and so it took more energy to motivate myself and get it all done.

I can imagine some people might find this frustrating though, or want something more concrete. You need to be honest with yourself about the extent of your itchy brain and how much it will drive you on to new things, and how rewarding you’ll find it to get that itch scratched (this metaphor is getting… disturbing). I think in this sense a PhD is a very good testing zone for working out how much you feel the urge to pitch your own ideas, compared to how much you like being given direction or structure. If you’re going to embark on a post doc career, it really really helps to prefer the former.

The key to any job is finding something you're good at.

The key to any job is finding something you’re good at.

4. Are you happy working for long term goals?(Some very long term. Like veeeerrrrryyyyy long term.)

I remember my driving instructor telling me that she used to work for a bank, and had done some kind of ‘work out your personality type’ training and realised she really valued tangible goals and task focused work with clear timelines. That had led her to instructing, which is a pretty neat choice – there’s a very concrete goal (pass the test) and you naturally work for specific periods on a specific task (with each new client you work through each stage of learning to drive, until they achieve the goal.)

I think she would have hated research. Projects take ages  – and not just from the point of funding to the point of completion. There is prep beforehand, and lots of work afterwards. They often overlap in messy ways – you can be writing up the results of one study into a proposal for another study. The timelines shift due to all manner of issues. If you thought your PhD was long, I’m afraid to say it only gets worse. Once you’re working on broad themes or on a body of work, you can very rarely identify where one bit started and another ends.

Similarly, seeing concrete outputs of your work can take years. Often the concrete stuff isn’t necessarily what you’re interested in though – I think on a personal level goals are more along the lines of “Helping instigate changes in X”, “Becoming respected at Y”, “Achieving a deeper understanding of Z”.  Personally, I’m happy with this. I guess I’m not much of a concrete thinker. For me, it’s about progress made overall, rather than any specific piece of work. I think if you’re considering a post doc, think about whether tangible outputs in a definite time frame are important to you. If yes, you might find a lot of post doc work frustrating rather than rewarding.

It's all about patience. Playing the long game. Your time will come.  *drool*

It’s all about patience. Playing the long game. Your time will come. *drool*

(NB. You are expected to define all these things as concrete outputs and explicit timelines in funding applications, and will most likely be expected to do so in a fellowship application. My points above relate more to the experience of being a researcher, not the skills you have to hone to fund being a researcher…)

Right. That’s my 4 big questions for deciding if you want a post-doc career in academia. Sorted?

Liking it really should be the deciding factor though. Liking it doesn’t necessarily mean liking all of it, or liking it all the time, or thinking some bits don’t need changing, but I think fundamentally enjoying research on some level is vital. The job market is highly competitive and there isn’t much in the way of additional bonuses, so you have to be keen to do it for its own sake and be willing to compete for it. In a way, fellowships are for those questionable folk like me, who genuinely can’t think of a more interesting way to spend their time. If that’s you, then go for it.


Posted in Academic Stuff | Tagged , , | 1 Comment

Add To Cart? What online shopping can teach us about patient-centred services.

Mark Duman, from Monmouth Partners and PIF Online, gave an excellent seminar at our Centre earlier this week. He made lots of very good points, about what it means to be an active patient and why the NHS doesn’t – but should – encourage patients to adopt this more active role, but one point in particular I want to focus on is the ‘Amazon analogy’. He made the point that we are active consumers in terms of most services, particularly online, but when it comes to healthcare we sit back into a passive role.

You might think online shopping robs us of jolly japes such as this, but the camera doesn't show the woman with the rifle to the far left who *really* wants that hat... There were no survivors.

You might think online shopping robs us of jolly japes such as this, but the camera doesn’t show the woman with the rifle to the far left who *really* wants that hat…
There were no survivors.

I started thinking about how I typically buy a product online, and what expectations I have. I search for the item I want. If I’m on a specific site, I even get slightly annoyed if it doesn’t recognise what I’m typing part way through and take me straight to it. I’m shown a selection of pictures, the highest and lowest price available from different sellers, the time it would take to dispatch, and a 5 star rating based on other’s feedback. I quite often read the reviews when I purchase online – there’s always one or two that are ill-thought out and a bit useless (“I ordered it in blue but when it came it wasn’t like the blue that I wanted what a rip off” etc) but on the whole it’s a good way of checking what the product is like in reality and quite often other people will be answering the questions that matter to me (“the wire is quite long enough to stretch round the average computer screen” for example). I then choose which seller to buy from, based on the cost, delivery time and their feedback.

But it often doesn’t end there. I expect a confirmation email of the order and also the dispatch date. I like another email when it’s actually sent out and a clear message saying who to contact if there’s any problem. If I’ve purchased directly from a store, rather than from another individual on ebay for example, I can choose for them to deliver to my house, a different address or a click and collect spot. I now usually expect them to take my mobile number and call me when they’ve arrived with the delivery, not just leave a soul-destroying depot card. If I receive the item and it isn’t right in some way, I expect clear instructions on how to return it and a full refund. If months later I want to check something about the order, I type it into the search box under ‘My Account’ and can see everything I need. And if I actually only ever got to the stage of looking, I will often get emails from the store suggesting similar items.

Now you could just focus on one aspect of this – greater choice. The ability to ‘shop around’ and compare, and pick which store or buyer to purchase from. Certainly the ConDem government have focused on this in their “reforms”, with choice of provider somehow in their minds key to patient satisfaction. But there’s much more happening here, and I think focusing only on choice is taking the idea of patient as consumer too literally when there are other aspects of patient need that we could explore.

There’s linking in with other customers to share knowledge and experience.  There’s personalisation, in terms of choice of how the product gets delivered and personalised recommendations based on my buying history. There’s a wider service offered beyond the sale, taking into account delivery and returns. There’s reassurance. There’s control, on my part, and I’m kept fully informed and able to check up on the order whenever I want. I actually tend to use the same stores regularly even if Google shopping shows the item is slightly cheaper elsewhere. This is about experience, and perhaps trust: I know the quality I can expect from those stores and that there will be no problems with any of the other issues, and I value this sometimes over and above saving a few quid by buying from another, cheaper, seller.

It’s in these comparisons that health care really falls down. Can we share our experience with others (sites like Health Talk and Patients Like Me show how powerful this can be)? Can we access our own information? Can we opt for the service that’s most practical for us or do we have to always do the equivalent of taking an afternoon off to go and wait in the depot queue until a grumpy postman insists they never had your item anyway. In terms of the wider context, it made me think about parking. One of the most common complaints about hospitals is that there’s never enough parking, and I’ve seen clinicians on twitter once or twice ask that patients please remember they’re getting top quality health care and so maybe don’t just complain at them about the parking when you’re getting your treatment. But this ignores the fact that, for the patient, getting a parking space is part of that health experience, and it can be stressful and upsetting when the patient is probably already quite stressed and upset. I’m not expecting every GP or surgeon to become intimately involved with parking arrangements. But they should at least recognise that these wider issues are part of that patient’s journey and affects how they feel. Also, maybe if the consultants kept bugging the managers about better parking for their patients, something would actually get done about it…

S'alright mate, I'm just nipping in for open heart surgery, back in a jiffy.

S’alright mate, I’m just nipping in for open heart surgery, back in a jiffy.

I think this post makes a similar point about digital services more broadly. It’s easy to just focus on one ‘dimension’ of user need, when actually we need to move out into different areas. In the case of health care, choice is just one dimension, and stretching out along it to give more and more choice isn’t really development and it’s very easy to neglect the other areas of user need that we need to extend into. The online shopping contrasts show how community, control, communication and care are all other dimensions of need that health services often fail to meet.

I can imagine there being a service like this for research. You could log on, search for projects you might be interested in, and choose how to take part. The research team then sorts out the ‘delivery’ to you and makes sure you’re fully informed about the project and how to get more information about it. I’ve seen calls for databases like this to be put into action, and arguably it’s what sites like Citizen Scientist are aiming for.

The one problem with this though is that it assumes people will think to log on and look up a research study. Maybe health research should be more like Kickstarter. Kickstarter for those who don’t know is a ‘crowd funding platform’ – rather than products being available and you buying them, you read a description of what the product would be, and if enough people promise to buy it then it gets made.

Imagine a research service where we tell people about a project we want to do, say a research trial. We advertise it and reach out to the different communities, digital and offline, that we think could be interested. We ask for their support. They can get involved at multiple different levels depending on their level of interest and how much they can throw in (in terms of time in this case rather than money). They get regular updates about the progress of the project and are quickly informed about when they will get the goodies they asked for. More subtly but very importantly, they feel a part of it. They made it happen.

If we took this analogy the whole way, we’d say that research projects wouldn’t actually get funded until enough patients back them. I imagine this very idea would scare the hell out of the vast majority of researchers. But I’ve gotta admit, I’m intrigued…

Posted in Thinking about research | Tagged , , , | Leave a comment

Attack of the Mind Reading Robots?! Not quite yet…

Polygraph machines are perhaps the most well-known technology that have labelled themselves as ‘lie detectors’, able to read whether people are hiding the truth. They’re terribly popular, at least among people who don’t concern themselves too much with whether they actually tell the truth (looking at you, ‘scandalous’ daytime TV shows.) The problem with polygraphs as a lie detecting machine is well established though, and beautifully illustrated here by Vaughan Bell who recounts the case of Buzz Fay, who was wrongly imprisoned for two years based on a polygraph, studied the machine, and then managed to teach other offenders how to beat it.

It's just like a Love Is... cartoon.

It’s just like a Love Is… cartoon.

Polygraphs work by comparing your level of arousal, as measured by blood pressure or pulse, when asked certain questions compared to a baseline established when you answer questions known to be true. This is why polygraph scenes on’t telly involve the first few questions being obvious things like “Are you sat in a chair?”, “Is your name Detective MacBreakstherulesbutdammithegetsresults?” and so on. This establishes a baseline of your resting level of arousal, and the polygraph administrator then looks for spikes of arousal when asked questions like “Did you sleep with the Chief’s Daughter and never call her back?” or “Are you Keyser Soze?”. But if you force yourself to get stressed during the baseline – say, by imagining watching Jeremy Kyle – then your baseline level will be inflated, and the machine won’t see any ‘spikes’ when you react to other questions. The idea that polygraph machines are lie detectors is therefore misleading – they’re just a machine that takes measures, and open to manipulation from those who understand how that machine works.

Implicit attitude tasks  are another tool that has been occasionally heralded as a lie detector. These tasks involve showing you different pictures or words in a category (say “women” or “men”) on computer and asking you to hit a particular key when you see one of them, then also asking you to hit a particular key when you see a “positive” or “negative” term. The researchers then look at your reaction times, and check if, for example, you were quicker when “female” and “positive” were on the same key, which is taken as showing an implicit positive bias toward women.  The idea is that subtle but measurable differences in your reaction times will show what you really think.

It really livens up the stale Dinner Party format. He's asking "Did you delibertaly pick Joe's keys out of the bowl last time instead of mine?" Via

It really livens up the stale Dinner Party format. He’s asking “Did you deliberately pick Joe’s keys out of the bowl last time instead of mine?” Via

There was much talk of the potential of such tasks to work as lie detectors, with studies for example showing that people who would self-report no racist bias would then be shown to “implicitly” prefer white people. Over here at Project Implicit you can check your own implicit biases toward race, gender and much more, which sounds like ounces of fun. I even recall these tasks being suggested as a way of identifying paedophiles, if you measured their reactions to pictures of children and found they “implicitly” preferred suggestive images of minors. The problem is that, just like the lie detector, the test is pretty easy to fool. If you just react very slowly to every single picture, then this makes it harder to detect those subtle speed differences that occur when you’re trying to react as fast as you can.

This brings us to the inspiration for this post, an article in The Conversation about how a combined set up of penis-arousal-measury-thingies (ok, apparently it’s ‘penile plethsymography’ or PPG),and virtual reality technologies can help identify sexual offenders. It works by tracking both how long they look at virtual reality images of minors and also their physiological arousal to those images.  The article says:

[The researchers] used headsets that track eye movements and record how long participants spent gazing at images. They also measured participants sexual arousal through penile plethsymography (PPG), which measures the flow of blood to the penis.

Combining PPG and virtual reality to gauge the behaviour of sexual offenders in the past has been criticised because of the possibility that they game the system by simply not looking at the images. The eye-tracking capability of the headset overcomes this problem, recording not just which computer-generated images of adults or children the participants view, but over which areas of the body their gaze lingers.

The article itself focuses on various interesting issues around the virtual reality aspect, like immersion, but for me this classic argument of getting past deception to the truth is the most interesting part –  and as with the cases above I’m not convinced this could do it. I’d need to see evidence that people who are instructed to, and who then really try to avoid looking at images that they find arousing, can’t play the system in some way. I can imagine this being through a combination of the lie-detector technique (just think of arousing things the whole time and there won’t be any clear differences between stimuli) and the implicit task dodge (look at everybody the same amount of time and again screw up the comparison data.) Maybe the headset/ PPG makes this harder to do, but I’d question whether it makes it impossible. It’s also of course one of those cases when you can’t see how it would work in practice. It would presumably not be ethical or legal to just try it on anyone – especially with the invasive aspects, you’d need good reason to assume the person has these attractions, but if you have good reason anyway then what does the headset add? Does it tell us something above and beyond, or more reliably, than the other forms of evidence we might use?

It strikes me that “Hey, we can see what people REALLY think!” is a pretty standard go-to for any devices or procedures that want to show they have special power or insight. I guess because the idea of reading minds is so appealing (to some. I personally think it would be hellish…) and it seems an easy route to demonstrating value – This Machine Reads Minds. We can spot liars, expose racists and catch sexual offenders. In reality, I think these techniques rarely achieve these feats, and people continue to be more complicated and harder to read than any tool can tell us.

Sadly mind reading is not just a technological step away...

Sadly mind reading is not just a technological step away…

Perhaps more interesting to me in all this is what it says about our mixed up relationship with the issue of body and mind. I think lots of people still see body and mind as completely separate. Certainly this would go some way to explaining why “physical” and “mental” health get treated so differently, or why some people think you can separate illness into “real” or “psychological”. I wonder if the above ideas fit into this in some way:, the appeal of somehow reading those pure signals from the body like heart rate, reaction times, eye movements, the ‘tells’ that will reveal what the mind is really up to.  We believe that what the body physically tells us must be ‘real’ or true, not like that chatter from the mouth full of the mind’s distortions and lies. Consequently, the idea of machines that can pick up on these bodily confessions, and save us from the perils of just talking to people, have huge, but perhaps undeserved, appeal.

Posted in Technology | Tagged , , | Leave a comment

Sharing is Caring: On trial data sharing in health research

I’ve been following the #EHILive updates today on twitter about the future of digital health.  There seems to be a lot of attention on the issue of open data or data sharing, and the potential of this to gain new insights and get the most out of all the data on health that is collected. From a research point of view, this chimes with debates being had around whether trial data sets should be ‘publicly’ available and accessible (Note that this is slightly different to calls by eg. #AllTrials which argue that all findings are made public, but not neccesarily that the original data sets have to be shared – it’s discussed here under point 4). The benefits of using original data are that the analyses we can do are much more robust, and we can interrogate the data sets in greater detail than we can do when using summary results. For an overview, I described here the benefits of what are known as ‘individual patient data meta-analyses’ for answering important questions in health research. But let’s talk about the data itself…

If writing your own blogs isn't excuse enough to post dubiously relevant Star Trek gags, then I don't know what is.

If writing your own blogs isn’t excuse enough to post dubiously relevant Star Trek gags, then I don’t know what is.

Currently, trial data sets are the property of the Chief Investigators (CIs), or someone else attached to the trial, such as an academic Trials Unit, rather than being stored in an open repository. This means if you want to use the data, you have to contact the individual trial teams and essentially try to pry it out of them, and some can be more helpful than others. It typically requires a merry-go-round of emails with the original authors, trying to work out what was measured where and how. A study by John Ioannidis and team in 2002 estimated that putting together a combined set data of multiple original studies involved 2088 hours of data management and approximately 1000 emails between the original authors and the study team  –  very costly and very inefficient.  With shared data,  the idea is that by asking trial teams to put their completed data set into a central system, researchers will be able to more quickly  get access to all this data to answer those important questions, rather than data sitting in isolated silos where it can’t be reused or taking up vast amounts of time just to collate in the first place.

This argument definitely has its critics however. There is firstly a pragmatic argument – that organising data sets for open use is an additional drain on time and money. Personally I don’t think this a sufficient reason to abandon the initiative anyway, and also I think although there would be initial costs, it should be possible in time to set up an efficient and easy to use system for submitting, storing and retrieving data.

As someone who has worked on secondary analyses, I can see further benefits in encouraging people to make their data set accessible to outsiders, for example making variable names clear and obvious, as often trying to navigate someone’s data set is akin to rustling through their wardrobe- simultaneously invasive and frustrating (“Why did they put that there? What even IS that?”).  I think having clear templates for how data should be stored and labelled would be much more efficient and helpful to anybody (including the original authors) who might want to revisit the data at a later time. On a different note, I can imagine how such templates could actually help standardise analyses and reporting. It would be possible for example for anybody to check the original data, to see how the data was handled (was missing data accounted for? Did they check if assumptions for the statistical analysis were met?), and for reviewers to check if data sets have indeed been organised according to the plan made in the study protocol.

Sure, you can have access to my data. IF YOU CAN FIND IT MWHAHAHA.

Sure, you can have access to my data. IF YOU CAN FIND IT MWHAHAHA.

There is another reason some people balk at the idea of data sharing though – it’s another level of scrutiny, with potential for criticism and embarrassment, and I think the idea provokes a certain kind of academic paranoia (” How dare they think I’m wrong! OHMIGOD WHAT IF I’M WRONG”). Dorothy Bishop nails these issues in her excellent post here, which acknowledges that data sharing can be scary, but points out that this in the end can surely benefit researcher and researchers by encouraging us to be more stringent about our analysis and fastidious about how data sets are compiled.

In chats over coffees/pints with colleagues though, it doesn’t seem to just be the extra work or the extra scrutiny that puts people off. Some people refer to issues of ‘ownership’, with the argument being that if they put all the leg work into getting a trial funded, setting it up, collecting the data, etc etc, then it isn’t fair to suggest they hand over that data to a public repository where Johnny Nicks-A-Lot can access it, do some analyses, and publish it for their own glory. It’s a “Bloody secondary analysts, comin’ over ‘ere, stealin’ our data” kind of argument.

I really don’t buy it. For one thing, those original trial researchers will probably have already published their own analyses, and I think it would be perfectly fair to say that data isn’t made open access until the study team has finished with it themselves, so no-one can swoop in and steal credit prematurely. This might include doing multiple analyses themselves, which is fine, and I think this would just emphasise the need to state that they’ll do them in the protocol (to avoid fishing expeditions.) They’re not being robbed then of any opportunity they could have had. I also think it could become a further award in itself – those who do submit their data sets to a repository should be acknowledged and applauded for doing so. 20 years from now, I can imagine on your CV, under the bit about impact factor or amount of grant money won, you might have a comment about how many data sets you have contributed to public repositories, and perhaps even a quantitative figure (like the impact factor) demonstrating how much additional work your data set has supported.

Finally, I have a more personal issue with “nicks-a-lot” argument, which I boil down to: IT’S THE PUBLIC’S DATA, NOT YOURS. These trials are often publicly funded, worked on by publicly funded research staff, and members of the public will be the people providing all that data. The idea that at the end of all that a lone researcher can say “S’mine!” and potentially stop further important studies being conducted makes me cringe.

Data is unimpressed by arguments against data sharing.

Unimpressed Data is unimpressed by arguments against data sharing.

The last issue for me really is key – the fact that isolating data sets means we’re robbed of opportunities to get the most out of them, to do big, important studies using hundreds and hundreds of patients, which could make a real impact to health care. Hence the title of this post – sharing is caring, if we care about the impact we make on health care rather than our personal ownership rights, and if we care about making sure all data sets contribute everything they have to offer, not just to papers for our own CVs.

I remember reading once that the drive to better structure how we collect, store and access data is about building ‘an information architecture’. As a data geek, I find this a beautiful metaphor. Architects, after all, aim for efficiency and elegance in their design, both things that typically appeal to scientists. I think data sharing could be a fantastic opportunity at developing this architecture, building something that is both accessible and usable. What my conversations with colleagues show however is that the challenge to this may not be in  designing the building itself, but the fact that some researchers still want to lock all the doors.

How can we design a data architecture that lets people in?

How can we design a data architecture that lets people in?

Posted in Thinking about research, Trials | Tagged , , , | Leave a comment