Category Archives: Tech stuff

There’s an app for that

After much complaining it is great to be able to be positive about the NHS ePortfolio. Developer Ben from NES has been working on an app which is now available in beta version. It is html5 so whether you’re an apple or android user it should work on your device. You can download it right now, have a go and provide feedback to make sure the next iteration is even better.

Go to this site on your device: app.nhseportfolios.org  and login with your usual password.

At the moment the functionality is limited. It supports reflective logs and ticketing, but does not allow offline completion of assessments such as mini-Cex. I also could not get it to store data within the app to synch later initially as it made me synch before I could logout, or data was lost. I also got stuck inside the app and couldn’t get back to the rest of my iPhone. This was a bit of a nightmare – stuck in the ePortfolio forever! It would be great to hear if anyone else had these problems or if it was just me! I have since managed to use it without crashing – phew! 

You can send your feedback to:  app@nhseportfolios.org 

An app is long overdue and does not fulfil all of our ePortfolio dreams yet, but it’s a start and should be celebrated. Hopefully, by testing and getting feedback before further work, the app will fulfil the needs of users and make logging evidence of our skills and learning experiences fit the realities of our working lives. And perhaps this agile way of working might permeate back to the rest of the ePortfolio. A girl can dream…..

Open Badges in MedEd?

Conversations started online lead to all sorts of places. Today they led me to Leeds School of Medicine to learn about Open Badges in Healthcare Education.

There’s a lot to say on this subject so please follow the links to learn more. I have tried to summarise some of the key opportunities and challenges and how they relate specifically to doctor’s ePortfolios and Postgraduate Training. This is not because Open Badges are limited to this context, but because this blog is focused on these issues. Overall my personal opinions are:

  • there is a need for a better method to collate and accredit learning, particularly informal learning. CVs are limiting
  • this is not a threat to established models of accreditation, or to institutions, but a complimentary system
  • Open badges have great potential as a mechanism of celebrating excellence and supporting medical trainees in differentiating themselves
  • as with all technology/software we need to be clear about what it can and can’t do, and not expect a panacea
  • as doctors we are currently limited by the fact that existing systems do not interface with other systems. No API = no possibility for progress. The whole point is that open badges cut across multiple systems and contexts, so they must not exist in yet another silo, separate from eg the ePortfolio. We need progress in this urgently
  • the drive for these new ideas and opportunities will come from the community, not institutions. Most institutions are likely to be sceptical initially but catch on later once the principle is established (examples would include Hospital Trusts, Royal Colleges, LETBs, Universities…). I would love to be proved wrong about this!

The Concept

Open badges are a mechanism for accrediting learning that happens everywhere. They are more than a graphic that you slap onto your online profile. The image has inbuilt into its’ code some essential data:

  • criteria for attaining the badge
  • the issuer
  • the earner
  • link to evidence

Doug put my picture into his presentation, so I’m returning the ‘favour’ You can find out more about him here: http://dougbelshaw.com/blog/about/

This is important as this means they can’t be simply copied or faked, and there is a way to probe what the badge means by checking the criteria and evidence. Much more information is on the Mozilla wiki, site, and collated blogs.

Future developments such as the ability to encode verification and endorsement mechanisms will be essential to win over the evidence-focused and reputation-aware landscape of healthcare education.

Doug’s presentation from today is on slideshare.

Opportunities for Postgraduate Training

Learning does not just happen in lecture theatres. We know this and demonstrate it every day. As medical students and doctors we piece together our education and training from a diverse range of informal and formal providers and contexts.

Last week I had many learning opportunities: I sat in on a specialist clinic, attended a Radiology meeting, completed an e-learning module, read several articles through UptoDate and pubmed searches, read a blog, contributed to debates on Twitter and followed links from tweets to journal papers, went to a seminar, and had a case discussion with a senior colleague. Trying to log all of this in existing tools is a challenge. My clinical NHS ePortfolio is certainly not flexible enough to facilitate this, and is unlikely to ever be, since it’s structure is dictated by committees of people who don’t use it. But the core system doesn’t have to fulfil every need if it interfaces with other systems. Open Badges could fit into this model very well.

Things that could be badged in medical education (a small and not at all exhaustive list):

  • attendance at a teaching session (1 badge), attendance at 80% of all ‘mandatory’ teaching sessions in your trust (separate badges build up like cheeses in trivial pursuit to unlock ‘St Elsewhere FY teaching attendance badge’)
  • contribution to #twitjc as a one off (1 badge), on a regular basis (higher badge), as host (super-badge)
  • completion of an online e-learning module (1 badge), completion of 15  modules relevant to your training programme (build up to unlock higher badge – BMJ already follow this principle with Silver/Platinum user. Wouldn’t it be great to draw other providers together along similar principles?)
  • giving a peer-led teaching session (1 badge), running a teaching programme as a result of an identified need (higher badge), having an impact ie safer handover/fewer bacteraemias/better compliance with antibiotic policy (super-badge)
  • completing generic mandatory training modules – many of which have been moved online (eg hand-washing, manual handling, equality and diversity), which could then be transported to a new Trust. A junior doctor could show their badges and not have to repeat generic training when they move jobs at 4 months, and instead spend time doing locally-specific training and then meeting the team (this was discussed at the workshop – great idea that I do not take credit for!)

Challenges

Many of these are discussed in more detail on the Mozilla pages (with additional technical, pedagogical, conceptual and philosophical considerations).

  • Equivalence: is my equality and diversity training module at St Elsewhere NHS Trust equivalent to your E&D module at St Somewhere Else? And will you accept is as such? Who decides?
  • Fake-ability: someone could set themselves up as a fake version of Highly Prestigious University and issue badges in their name. Future developments of verification (eg verified accounts) and endorsement would help with this
  • Over-exposure: If you can badge anything, does this devalue them as a currency? Not necessarily. There will be hierarchies of badge, just like there are hierarchies and power structures in other spheres. A peer-issued badge for ‘general awesomeness‘ would be value-less for a job application and no badge earner would choose to display this badge in that context. In contrast a University issued badge for ‘significant contribution to bedside Undergraduate Teaching’ or a Trust issued badge for ‘leadership in quality improvement’ would be worth significantly more.
  • Validity: What if someone earned a badge for hand-washing, but couldn’t then demonstrate the skill? Theory does not necessarily transfer to practice, but this is not a problem of open badges themselves, but of all teaching and learning
  • Admin support: the creation and issuing of badges needs thought and planning. It does not have to mean lots of additional work but of course would require initial startup resources. The discussion of beenfits vs costs needs to be fleshed out, and systems worked out to make badge issuing something people feel supported in doing, and will seek to do. The guys from myknowledgemap and reallymanagingassessment tools have thought about this already, and done a lot of the work. They have envisaged ways to support less code-literate mortals (like me) in the process. We can learn from their expertise, share the work, share costs, share, share, share….

What’s inspiring about the Mozilla Open Badges project is the level of transparency  and collaboration. The Mozilla Foundation is a global non-profit whose only motive is to “promote openness, innovation and participation in the internet”. This open-ness is incredibly powerful. They share all their code and processes, which allows others to create widgets and plug-ins and build layers of code/systems on top of their core system. This does not challenge or threaten Mozilla. It enhances their product and makes it more attractive and useful. This is a general lesson we can learn and apply to other contexts. The most obvious example would be NHS Hackday. Imagine what we could do with an NHS ePortfolio API (for a start we could already have built an app)! Imagine if Open Badges interfaced with the NHS ePortfolio! That would be one way to gather all the informal (and formal) lifelong-learning that is already taking place, and strongly encouraged by Royal Colleges, the GMC, and other healthcare bodies – from medical school entry to retirement.

I think an important point is that Open Badges are not a gimic. They are not a niche internet project. They are a response to a much bigger challenge to traditional learning. You may not think they are the right response, but they should be seen in this context. There has been a shift in how and where learning occurs which is challenging but positive. Better informed, engaged and motivated clinicians provide better patient care: what we’re all here for in the end. We need systems that facilitate and celebrate learning in new contexts. As learners we yearn for it it. As institutions we have a duty to support it. As patients we should welcome it. Open Badges could be such a system.

Thanks to Leeds School of Medicine for being so open and welcoming; to myknowledgemap for being so positive about collaborating; to Tim from NES for putting up with my ranting; and to Doug from Mozilla for his pragmatism,  enthusiasm and expert workshop facilitation. 

Progress

Things have been a little quiet on the site lately, and you would be forgiven for wondering whether I have lost interest in the NHS ePortfolio. You may have started to think that nothing is being achieved, and therefore not bothered to comment or contribute to the discussion.

Don’t believe it!

Progress may be slow, but real change takes time. Quick fixes are great and can have a major impact on functionality. Remember the problems with being unable to link to multiple curriculum items? Fixed! See the tech improvement shopping list for other modifications that have already happened. Quick fixes are also visible and keep up enthusiasm and morale. But they don’t address the route causes of problems and don’t change systems. Changing systems takes time.

A major breakthrough has been the creation of an ePortfolio reference group at the Royal College of Physicians. You can apply to be a member of this panel and get your voice heard directly by the College. Not a member of the RCP? Then ask your own College if they have a user group you can join. And if they don’t have one, ask why not. The systems imposed on trainees are currently not fit for purpose, and we need to make sure people in charge understand this.

Dont forget that NES, the group who run the NHS ePortfolio used by Physicians, Paediatricians and others, is holding feedback events. I’m sure that this is in no small part due to pressure from this site and discussions on Twitter. If you can, go along and make your voice heard.

People are listening.  I have meetings coming up with the Royal College of GPs who use a different ePortfolio system but share common needs. I am also having a follow-up meeting at the Royal College of Physicians. We must clarify the commissioning and costs of the ePortfolio in order to collaborate across Colleges and effect change. When money is scarce we need to make it go further. The Academy of Medical Royal Colleges Trainee Doctor Group are gathering data on the systems in use by all specialities which will be essential is informing this discussion.

It could be so much better! Please keep up your support.

An ePortfolio wish list – created by an NHS ePortfolio NES developer

There have been lots of great suggestions on this site as to features that would make the NHS ePortfolio better. There has also been some insightful debate on the model that we should propose for the future, from aesthetics to functionality to commissioning. When I find a free hour I will make a summary page, as the debate has extended far beyond my initial expectations, and on arriving to the site it may now be difficult to see a coherent narrative. Some ideas for improvements are summarised in the tech improvement shopping list, others are amongst the comments on various posts.

A new development is that an NHS ePortfolio NES developer @ben555 has started a wish list where you can suggest improvements and vote them up or down the list.

Make a (NHS ePortfolio-related) wish

Whilst debate over longer term issues  continues, this is a great way to gather momentum for change. I have voted for an app – this is urgent!

Make your suggestion and cast your vote now…

NHS Hackday and the ePortfolio Data Liberation Front

I spent this weekend in Liverpool at NHS Hackday. I had no idea what to expect. I had never met anyone there before and only knew a few names from twitter and google groups conversations in the weeks running up to the Hackday. I wasn’t completely sure I knew what a Hackday was.

I was astounded. 

I spend a lot of my life getting frustrated by the slow pace of change and the massive inefficiencies in the way that we work. I want to be freed up to spend time teaching, learning, writing, thinking, talking to patients and providing care. I hate unnecessary paperwork and bureaucracy. I hate meetings that don’t achieve anything.

NHS Hackday was a breath of fresh air. A diverse group of people with totally different backgrounds, most of whom had never met,  got together, discussed problems and solved them. In a weekend!

I will describe here what happened at the Hackday, what our project “The NHS ePortfolio Data Liberation Front” achieved and why it won 2nd place. There is far more info about how it is run, by who, and why on the NHS Hackday site. You can also see an interview with Carl (from OpenHealthcare) on Youtube:

 What follows are my personal impressions.

The HACKDAY CoNCEPT

On Saturday morning, whilst people were registering and getting coffee all those with ideas for projects wrote them on a board. Everyone gathered in the main hall and each idea had 2minutes to pitch. After all the pitches, people gathered around signs indicating each idea, and people formed groups. Then the work began. Groups discussed their vision, their proposed solution, and thrashed out conceptual and technical details. Fuelled by enthusiasm, tea, coffee and wotsits, software developers created things out of thin air (OK, out of data and code, blood, sweat and tears). Health professionals like me, who couldn’t code, were on hand to give context to the projects and point out real-world hurdles, which could then be worked around.

The NHS ePortfolio Data Liberation Front

Our group consisted of me (full of ideas, no understanding of code), Nicolas Tollervy, a developer (a genius with lots of patience and an incredible ability to work round every problem the project presented him with)  and Marcus Baw, (a GP who can code a bit and is a RCGP Health Informatics Group member, who was a great bridge and font of knowledge on NHS informatics issues).

We discussed some of the problems with the current NHS ePortfolio and possible workarounds. Since the code is not open and there is no API this was no simple problem.

We discussed the urgent need for an app to make trainees and trainers lives easier, and make WPBAs educationally valid. Any app would have to be able to get data into the ePortfolio so that a WPBA showed up not just in the personal library section as any random document, but in the WPBA section. With no code and no API this would be a great challenge.

We decided to focus on the fact that my data is locked in a vault in my ePortfolio. Whilst it is in there I can do nothing with it.

I want to liberate it, as I could then do anything I want with it! Ideas include:

  • visualise my achievements and progression
  • present the data in a way that my supervisor can see, understand and give feedback on
  • present the data in a way that makes it clear I have achieved all the competencies required by the JRCPTB for ARCPs and CCT
  • integrate the data into my CV, my online CV, an alternative ePortfolio (mahara, Googlios etc), use it for job applications
  • allow me to take the data with me into another role (progression or change of career path) eg Foundation Trainee –> Emergency medicine ACCS trainee –> GP trainee –> GP (all use different ePortfolio systems)

Not only is there a practical need for this, but the more we talked about it the more I realised that this is bigger than practicalities. It’s a philosophical argument. It’s my data. About me. I want it liberated. I can already download a PDF so clearly no-one disputes the fact that the data is mine and I have a right to it, but a PDF is useless.

@ntoll worked incredibly hard (with breaks for coffee, sandwiches, a trip to the pub and a curry house), came up against many problems and found ways around them all. We modified our plan as we went along, and decided that the best use of our time would be to do a ‘proof of concept’ and focus on a particular data set within the ePortfolio (there’s a lot of data in there, and it’s not organised as logically as you might imagine!). By the time we reached the submission deadline of 12.00 on Sunday we had something to show for our efforts. @ntoll made some finishing touches and we put together a brief presentation.

All 15 projects that had been selected from the pitches presented (a strict 5min and 1min for questions) to a panel of judges including: @MarkPriceDavies (chair), Ian Gilmore, Dr Farath Arshad, Zeinab Abdi, Francis Irving @frabcus, Dan Lynch @MethodDan, and Lilian Wiles. They deliberated and at 17.00 announced the winners.

The Other Projects

You can see more details of the projects on the NHS Hackday site, and get all the code through the wiki and on github, since all projects are open and shared. There were lots of fantastic projects but those that particularly caught my attention were:

  • AskIt (a general purpose question asking android app for any questionnaire you need – Waterlow, MUST score, falls assessment etc. Simple, effective, important!)
  • Making sense of patient comments (data visualisation from sources such as NHS Choices – massive potential applications)
  • CoIncidence Gate: a Conflict of Interest tool (scraped data from conflict of interest statements on Pubmed – something like 480,000 papers analysed!! Again, follow the link for more discussion on the massive potential applications of this project)
  • BleepBleep (making in-hospital communication better. An end to having to call switchboard. An end to the bleep! I trialled this, and am keen to help get it into hospitals now! Stop wasting time on hold)
  • GAAG: Guidelines at a Glance (there are well-studied barriers to doctors using guidelines, meaning patients don’t get best care. GAAG provides quick access to personalised most-used bits of guidelines on an app. Lots of potential for social add-ons, highlighting when guidelines change, seeing what peers use, rating bits of guidelines. See presentation for more info. Can’t wait to use it!)
  • Bloodcount (haematologists sit at very advanced microscopes counting different normal/abnormal cell types using very un-advanced technology = clicker and pen and paper. Bloodcount is a desktop system of a counter with keyboard shortcuts, reference normal and abnormal cells, report generation and learning function. Hard to describe to do it justice. A worthy winner!)
  • wtfdoc (an NHS jargon buster for patients and relatives as an app. Has a database, and if a term is unknown it will crowdsource answer through twitter and other sources. V clever!)

Why I think We Won a Prize

Our project won the First Scraperwiki prize for scraping, and came joint second overall on the day. I think the reasons we won are multiple:

  • @ntoll achieved amazing things writing novel code to scrape data out of a closed system and generate a .json file of hierarchical data that could then be used. In just a day and a half this was some achievement!
  • our pitch was powerful as this is an issue for all doctors of all specialities at all levels, especially with revalidation now a reality. Facilitating learning for healthcare professionals is in all our interests as a society
  • the concept of data liberation goes beyond this project. Who owns the data in public databases? Who owns the data in the NHS? What right does an individual have to their own data? What right does an institution have to keep it from them?

What next

I owe a huge thank you to the organisers, supporters, volunteers and participants at NHS Hackday Liverpool 2012.  And a special thanks to Scraperwiki for providing prizes including my beautiful new Google Nexus 7! This weekend I saw innovation in action, providing real, practical solutions to the day-to-day problems facing those who work in and use the NHS. Some of these solutions are now in use – today! Others will be worked on outside the Hackdays or at the next one. I have had my mind opened to new ways of working and have returned to work today full of enthusiasm and inspiration.

There’s no going back now. I’m a doctor who loves geeks who love the NHS, and I have the T-shirt to prove it. 

Can we make ePortfolio open source? a guest post from Karen Beggs

My first question is WHY?

Here are the main issues I hear about:

  • A lot of trainees aren’t happy with workplace based assessments
  • Internet speed is an issue in some NHS locations
  • Some people don’t like using an ePortfolio
  • Some people want to have more input into ePortfolio design
  • Some trainees want their seniors to be more engaged with their learning
  • There is a common misunderstanding that College membership fees are used solely to pay for the ePortfolio

So what are we already doing about these issues?

  • We are eliciting feedback directly from the wider ‘user’ community through social media to find out what usability improvements we can make…and get them done.

We’ve started this already… following a conversation last month with a trainee who was frustrated by the curriculum linking process, our architect made a simple change that was deployed a few days later (see demo here), reducing the number of clicks needed to make multiple links. We have also introduced a twitter feed, visible on the www.nhseportfolios.org home page).

We are moving to a more elastic hosting environment so that as the system gets busier it can engage more resources to deal with the increased load. We aim to have this fully implemented by autumn 2012.

The NHS ePortfolio team do not make decisions about assessment processes, training requirements or the use of specific workplace based assessments. Expertise in these areas usually lies with the Colleges, Postgraduate Deans and the Academy of Medical Royal Colleges. However, we can:

  • Help connect trainees with ideas (or complaints) with the relevant people, whether College, user group, developers or others. In July 2012 I attended the AoMRC Trainee Doctors Committee, and as a result will recommend to the Chair of the specialty ePortfolio User Group that a representative from the committee sits on that group. Some trainees are not aware of their own College decision making processes, and we will pass on contact details as required.

Would open source address any of these issues?

As far as I can see, NO!

But why don’t we just hand over the code to the large community of willing, enthusiastic OSS developers?

  • It’s not as easy as just handing out ‘the code’. Open source software must comply with a number of criteria (see www.opensource.org), many of which would contradict the current NHS ePortfolio license terms.
  • Who would fund re-writing and re-negotiating software licenses for the existing 25 or so organisations using the NHS ePortfolio? What if one of these organisations objects? It’s an integrated application with many shared features, so to separate out one ‘Customer’ would require a large-scale re-write. That seems to defeat the purpose.
  • The ePortfolio is integrated with a number of external (usually College run) systems and moving to an OSS model would have implications for each of these systems. Would Colleges want to pay to conduct a thorough risk assessment before signing up? And would they then want to pay for any adjustments needed to maintain the integrity of their own systems?
  • There would still have to be stringent controls over the quality of the code submitted. This would require a quality control team – possibly a larger one than we have at the moment. Who would pay for this?

I’m not sure I quite follow the argument… get rid of our current team of developers (some of whom have been with us for over 4 years), keep fingers crossed that some OSS developers can meet our commitments, beef up our QA team so they can check the code of the unknown OSS developers…. Seems that we increase our risks (of not meeting SLAs), decrease predictability (how can we hold anyone to a delivery date if we don’t employ them?) and end up with a QA team but lose our development expertise (the current team wouldn’t hang around for long – why would they?). I can’t see a sustainable business model in here unless we were to maintain a large core team – and if we do that, where are the assumed cost savings of OSS?

I have heard arguments that OSS is cheaper overall, but I don’t really see that cost is the problem (see My first question is WHY? above). It seems to me that the per capita charges for the ePortfolio are pretty reasonable. There is currently no charge made for any supervisor (educational or clinical), programme director, administrator, ARCP panel member or assessor using the ePortfolio. Per capita charges are based only on trainees at present. Would OSS have any impact on this? I can’t see that it would.

Final thoughts

If we were starting from scratch we would look at OSS as one of the options. We would probably look at an off-the-shelf ePortfolio too. We would be foolish not to. But we are not starting from scratch. We have an established, bespoke ePortfolio that is used across the professions (we have versions for Dentists, Nurses & Midwives, Pharmacists, Doctors and Undergraduates), is integrated with a number of external systems and capturing over a million forms submitted by ‘assessors’ every year. Each version has a custom set of features, making it adaptable and cost effective (sharing an underlying code base and database).

Many of the problems we hear about relate to complaints about the educational processes, and changes are already underway to address these (eg move to Supervised Learning Events in Foundation from August 2012). We contribute to these discussions when appropriate.

We have developed good relationships with our broad range of Customers, and continue to work with them to improve our change control and development processes. We work within the constraints of the NHS, which impacts our management of finance, procurement, stakeholder management, technology and decision making, as well as our governance arrangements.

We have an established application and an experienced team, whose expertise and commitment cannot be underestimated. Our development costs are at the lower end of the market, and maintenance charges are extremely good value. We can bring in additional specific expertise as and when we need to.

I can’t help but think the suggestion to move NHS ePortfolio to OSS is a solution to the wrong problem.

To Open Source or not to Open Source

Recent discussions around possible solutions to the need for NHS ePortfolio development have led to the suggestion that harnessing Open Source Software (OSS) may be the answer.

So what is Open Source Software (OSS) I hear you cry?

Open Source is collaboratively developed, freely available software or application. You may now hear people using the term ‘Open Source’ across a wide variety of different sectors in order to describe a more open, networked and user generated way of developing ideas and projects. While the term applied originally only to the source code of software, it is now being applied to many other areas.

But it is not just for new projects. For established projects, the software or application itself does not have to be dismantled.  There can be huge benefits of opening up the code to outsiders who then use the software, fix bugs, submit patches, file  bug reports, and create new content. Often for free. In times of ever-contracting training budgets and with austerity in the national consciousness, free sounds very appealing.

NHS Hackday is the most relevant example of this concept, and does a good job of explaining why OSS fits with the concept of openness in healthcare and academia, and why it is financially beneficial.  Examples of projects so far include an app to aid safe handover (in line with recommendations from the acute care toolkit, and clinical governance principles), and OpenBNF (an open source app for access to the tax-funded British National Formulary of medications, currently only available at a cost of £30 via a private provider). I think the projected cost savings may be over-enthusiastic, but the model of using the knowledge of “coalface” clinicians, and harnessing technological expertise for the public good is clearly powerful. This same argument could be made for the NHS ePortfolio as in the short-term all time saved means more time for patient care or educationally valuable activities. And in the longterm a better ePortfolio could facilitate better education and training, ultimately producing better doctors.

There are Open Source evangelists:

Carl is an evangelist,  Ben Goldacre is an evangelist

And there are those with legitimate concerns about control, and whether Open Source would deliver what is needed. As Karen Beggs (ePortfolio project manager at NES) points out it is no panacea: we must look critically at our needs and apply the right solutions. Here are some responses to questions about security and maintaining control, and an insight into the potential hidden problems of not using OSS.

Encouragingly, OSS in healthcare is not new (examples), and OpenSource in ePortfolios is not new (see Mahara) so there is already work to build on.

A vision of collaboration, openness, and harnessing clinical knowledge to create rapid solutions to real-world problems, working from the ground up instead of the top down is incredibly powerful and one the NHS and education communities should celebrate.

Will OSS be the cure for the NHS ePortfolio’s chronic disease? If you’re a geek who loves the NHS maybe you can help us find out..