“Study finds that health information on the Web is read less often than the same material on paper” What’s really happening?

This AMA news story by Pamela Lewis Dolan is a very surprising bit of news if you (like me) are improving outcomes by developing digital tools that engage patients.

Taking a closer look at the original research and it’s quite unsurprising that it’s based on a report of such low quality it genuinely surprises me that a Health Communication Journal would publish it.

Here’s why I think it’s completely invalid for the headline conclusion:

1) Apples Vs Oranges

The research contrasts engagement with web based and paper based information. Whilst it tries to accurately measure the clicks of users (once logged into it’s website) it uses a self report process for patients who were in the paper based group.

2) Importance of quality design of the web portal

Of the 170 recruited into the “web arm” of the trial the researchers reported that 20% faced a technical error that made it impossible for them to log on. I think this indicates that there were significant usability challenges with the web portal the patients were being referred to and as such it’s likely many of the other participants just gave up on sight of the poorly functioning portal (many of these probably went off and used Google, WebMd etc independently of the directions they had been given.

I think it’s fair to say that no one would be surprised if patient interest was lowered by significant reduction in the quality and style of paper based patient information so it’s a surprise that the researchers do not even mention this (despite the reported 20% outright failure rate of the portal)

I’m also quite sure that good design and user experience have a huge bearing on patient recall eg. the researchers reported “almost 70% of those who used the Web site did not remember using it 4 months later”. I’ve never met a patient who has interacted with a Doctor online (or via mobile) who has any trouble with recalling that experience even years later. A report documenting the encounter also has a big impact on this.

If you think I’m being unfair consider this of the 170 selected for the web study group the research conclusions are based on only 32 participants and only 6 of these logged into the website more than once:

Of participants randomized to the Web-based intervention (n = 130), 32 (24.6%) actually logged onto the Web site on the basis of tracking data. For these 32 participants, the majority logged on only once (n = 26)

> Participation not “Direction”

I think the finding that 4% were excluded “for noninterpretable answers” highlights the wrong attitude from the researchers who seemingly viewed online engagement as something patients should be “directed” to do rather than an opportunity for participation.

I think digital care providers should be aware that experiences should be designed such that patients aren’t ever in a position where they think they’ve done something wrong.

> Dud questions

I’m not surprised that patients gave noninterpretable answers as some of the questions were also nonsensical. Asking a patient who is signed in and looking at your web portal whether they received information about it’s existence is exactly the type of poorly thought through experience that has me checking out…

> Underestimation of patient web use by design

The report seems to ignore content searches made outside of the portal. From watching patients using the internet it’s common to see them opening a new tab or window to search for something. This obviously would be unrecorded by the design of this online portal.

Claiming only “28% visited an external website” might just be ignoring the 72% who just fired open another window and continued their browsing.

> Has the idea of logging on had it’s time?

Ending on a positive note I think the research most definitely highlights the categorical failure that happens when we require patients to log on. With a wide variety of online services such as Facebook, Hotmail etc patients stay logged in, and while I fully accept the privacy challenge that healthcare provides I can also see how mobile can help us move away from this antiquated pull model.

About David Doherty

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s