Tag Archives: NHS ePortfolio

2012 in review

The WordPress.com stats helper monkeys have prepared a 2012 annual report for me. Aren’t they nice?

Here’s an excerpt:

4,329 films were submitted to the 2012 Cannes Film Festival. This blog had 13,000 views in 2012. If each view were a film, this blog would power 3 Film Festivals

Click here to see the complete report.

This blog has had a lot of support this year. Thank you to everyone who contributed, commented, argued, debated, encouraged and supported it’s ideals along the way.

2012 was the year of debate about the purpose and functionality of the NHS ePortfolio. What will 2013 bring?

Open Badges in MedEd?

Conversations started online lead to all sorts of places. Today they led me to Leeds School of Medicine to learn about Open Badges in Healthcare Education.

There’s a lot to say on this subject so please follow the links to learn more. I have tried to summarise some of the key opportunities and challenges and how they relate specifically to doctor’s ePortfolios and Postgraduate Training. This is not because Open Badges are limited to this context, but because this blog is focused on these issues. Overall my personal opinions are:

  • there is a need for a better method to collate and accredit learning, particularly informal learning. CVs are limiting
  • this is not a threat to established models of accreditation, or to institutions, but a complimentary system
  • Open badges have great potential as a mechanism of celebrating excellence and supporting medical trainees in differentiating themselves
  • as with all technology/software we need to be clear about what it can and can’t do, and not expect a panacea
  • as doctors we are currently limited by the fact that existing systems do not interface with other systems. No API = no possibility for progress. The whole point is that open badges cut across multiple systems and contexts, so they must not exist in yet another silo, separate from eg the ePortfolio. We need progress in this urgently
  • the drive for these new ideas and opportunities will come from the community, not institutions. Most institutions are likely to be sceptical initially but catch on later once the principle is established (examples would include Hospital Trusts, Royal Colleges, LETBs, Universities…). I would love to be proved wrong about this!

The Concept

Open badges are a mechanism for accrediting learning that happens everywhere. They are more than a graphic that you slap onto your online profile. The image has inbuilt into its’ code some essential data:

  • criteria for attaining the badge
  • the issuer
  • the earner
  • link to evidence

Doug put my picture into his presentation, so I’m returning the ‘favour’ You can find out more about him here: http://dougbelshaw.com/blog/about/

This is important as this means they can’t be simply copied or faked, and there is a way to probe what the badge means by checking the criteria and evidence. Much more information is on the Mozilla wiki, site, and collated blogs.

Future developments such as the ability to encode verification and endorsement mechanisms will be essential to win over the evidence-focused and reputation-aware landscape of healthcare education.

Doug’s presentation from today is on slideshare.

Opportunities for Postgraduate Training

Learning does not just happen in lecture theatres. We know this and demonstrate it every day. As medical students and doctors we piece together our education and training from a diverse range of informal and formal providers and contexts.

Last week I had many learning opportunities: I sat in on a specialist clinic, attended a Radiology meeting, completed an e-learning module, read several articles through UptoDate and pubmed searches, read a blog, contributed to debates on Twitter and followed links from tweets to journal papers, went to a seminar, and had a case discussion with a senior colleague. Trying to log all of this in existing tools is a challenge. My clinical NHS ePortfolio is certainly not flexible enough to facilitate this, and is unlikely to ever be, since it’s structure is dictated by committees of people who don’t use it. But the core system doesn’t have to fulfil every need if it interfaces with other systems. Open Badges could fit into this model very well.

Things that could be badged in medical education (a small and not at all exhaustive list):

  • attendance at a teaching session (1 badge), attendance at 80% of all ‘mandatory’ teaching sessions in your trust (separate badges build up like cheeses in trivial pursuit to unlock ‘St Elsewhere FY teaching attendance badge’)
  • contribution to #twitjc as a one off (1 badge), on a regular basis (higher badge), as host (super-badge)
  • completion of an online e-learning module (1 badge), completion of 15  modules relevant to your training programme (build up to unlock higher badge – BMJ already follow this principle with Silver/Platinum user. Wouldn’t it be great to draw other providers together along similar principles?)
  • giving a peer-led teaching session (1 badge), running a teaching programme as a result of an identified need (higher badge), having an impact ie safer handover/fewer bacteraemias/better compliance with antibiotic policy (super-badge)
  • completing generic mandatory training modules – many of which have been moved online (eg hand-washing, manual handling, equality and diversity), which could then be transported to a new Trust. A junior doctor could show their badges and not have to repeat generic training when they move jobs at 4 months, and instead spend time doing locally-specific training and then meeting the team (this was discussed at the workshop – great idea that I do not take credit for!)

Challenges

Many of these are discussed in more detail on the Mozilla pages (with additional technical, pedagogical, conceptual and philosophical considerations).

  • Equivalence: is my equality and diversity training module at St Elsewhere NHS Trust equivalent to your E&D module at St Somewhere Else? And will you accept is as such? Who decides?
  • Fake-ability: someone could set themselves up as a fake version of Highly Prestigious University and issue badges in their name. Future developments of verification (eg verified accounts) and endorsement would help with this
  • Over-exposure: If you can badge anything, does this devalue them as a currency? Not necessarily. There will be hierarchies of badge, just like there are hierarchies and power structures in other spheres. A peer-issued badge for ‘general awesomeness‘ would be value-less for a job application and no badge earner would choose to display this badge in that context. In contrast a University issued badge for ‘significant contribution to bedside Undergraduate Teaching’ or a Trust issued badge for ‘leadership in quality improvement’ would be worth significantly more.
  • Validity: What if someone earned a badge for hand-washing, but couldn’t then demonstrate the skill? Theory does not necessarily transfer to practice, but this is not a problem of open badges themselves, but of all teaching and learning
  • Admin support: the creation and issuing of badges needs thought and planning. It does not have to mean lots of additional work but of course would require initial startup resources. The discussion of beenfits vs costs needs to be fleshed out, and systems worked out to make badge issuing something people feel supported in doing, and will seek to do. The guys from myknowledgemap and reallymanagingassessment tools have thought about this already, and done a lot of the work. They have envisaged ways to support less code-literate mortals (like me) in the process. We can learn from their expertise, share the work, share costs, share, share, share….

What’s inspiring about the Mozilla Open Badges project is the level of transparency  and collaboration. The Mozilla Foundation is a global non-profit whose only motive is to “promote openness, innovation and participation in the internet”. This open-ness is incredibly powerful. They share all their code and processes, which allows others to create widgets and plug-ins and build layers of code/systems on top of their core system. This does not challenge or threaten Mozilla. It enhances their product and makes it more attractive and useful. This is a general lesson we can learn and apply to other contexts. The most obvious example would be NHS Hackday. Imagine what we could do with an NHS ePortfolio API (for a start we could already have built an app)! Imagine if Open Badges interfaced with the NHS ePortfolio! That would be one way to gather all the informal (and formal) lifelong-learning that is already taking place, and strongly encouraged by Royal Colleges, the GMC, and other healthcare bodies – from medical school entry to retirement.

I think an important point is that Open Badges are not a gimic. They are not a niche internet project. They are a response to a much bigger challenge to traditional learning. You may not think they are the right response, but they should be seen in this context. There has been a shift in how and where learning occurs which is challenging but positive. Better informed, engaged and motivated clinicians provide better patient care: what we’re all here for in the end. We need systems that facilitate and celebrate learning in new contexts. As learners we yearn for it it. As institutions we have a duty to support it. As patients we should welcome it. Open Badges could be such a system.

Thanks to Leeds School of Medicine for being so open and welcoming; to myknowledgemap for being so positive about collaborating; to Tim from NES for putting up with my ranting; and to Doug from Mozilla for his pragmatism,  enthusiasm and expert workshop facilitation. 

My 15 mins of fame

I am having some brilliant and highly productive meetings about the ePortfolio this week, which I will write about as soon as I can.

In the meantime, you can see my 15mins of fame in the NHS Hackday video:

NHS Hackday 2012: Geeks who love the NHS

“It’s an amazing atmosphere when you can come with a problem as a physician and say ‘I know nothing about any of this stuff, but I know what I want to be able to do.’ And then to have a room full of people who have the know-how and the enthusiasm to go ‘we can do that.'”

I love @wai2k‘s quote, which really sums up the weekend and the project: “with collaboration between the people who create the software and the people who use the software you can potentially create something quite magical.”

The ePortfolio needs exactly this. We are getting there.

Can we make ePortfolio open source? a guest post from Karen Beggs

My first question is WHY?

Here are the main issues I hear about:

  • A lot of trainees aren’t happy with workplace based assessments
  • Internet speed is an issue in some NHS locations
  • Some people don’t like using an ePortfolio
  • Some people want to have more input into ePortfolio design
  • Some trainees want their seniors to be more engaged with their learning
  • There is a common misunderstanding that College membership fees are used solely to pay for the ePortfolio

So what are we already doing about these issues?

  • We are eliciting feedback directly from the wider ‘user’ community through social media to find out what usability improvements we can make…and get them done.

We’ve started this already… following a conversation last month with a trainee who was frustrated by the curriculum linking process, our architect made a simple change that was deployed a few days later (see demo here), reducing the number of clicks needed to make multiple links. We have also introduced a twitter feed, visible on the www.nhseportfolios.org home page).

We are moving to a more elastic hosting environment so that as the system gets busier it can engage more resources to deal with the increased load. We aim to have this fully implemented by autumn 2012.

The NHS ePortfolio team do not make decisions about assessment processes, training requirements or the use of specific workplace based assessments. Expertise in these areas usually lies with the Colleges, Postgraduate Deans and the Academy of Medical Royal Colleges. However, we can:

  • Help connect trainees with ideas (or complaints) with the relevant people, whether College, user group, developers or others. In July 2012 I attended the AoMRC Trainee Doctors Committee, and as a result will recommend to the Chair of the specialty ePortfolio User Group that a representative from the committee sits on that group. Some trainees are not aware of their own College decision making processes, and we will pass on contact details as required.

Would open source address any of these issues?

As far as I can see, NO!

But why don’t we just hand over the code to the large community of willing, enthusiastic OSS developers?

  • It’s not as easy as just handing out ‘the code’. Open source software must comply with a number of criteria (see www.opensource.org), many of which would contradict the current NHS ePortfolio license terms.
  • Who would fund re-writing and re-negotiating software licenses for the existing 25 or so organisations using the NHS ePortfolio? What if one of these organisations objects? It’s an integrated application with many shared features, so to separate out one ‘Customer’ would require a large-scale re-write. That seems to defeat the purpose.
  • The ePortfolio is integrated with a number of external (usually College run) systems and moving to an OSS model would have implications for each of these systems. Would Colleges want to pay to conduct a thorough risk assessment before signing up? And would they then want to pay for any adjustments needed to maintain the integrity of their own systems?
  • There would still have to be stringent controls over the quality of the code submitted. This would require a quality control team – possibly a larger one than we have at the moment. Who would pay for this?

I’m not sure I quite follow the argument… get rid of our current team of developers (some of whom have been with us for over 4 years), keep fingers crossed that some OSS developers can meet our commitments, beef up our QA team so they can check the code of the unknown OSS developers…. Seems that we increase our risks (of not meeting SLAs), decrease predictability (how can we hold anyone to a delivery date if we don’t employ them?) and end up with a QA team but lose our development expertise (the current team wouldn’t hang around for long – why would they?). I can’t see a sustainable business model in here unless we were to maintain a large core team – and if we do that, where are the assumed cost savings of OSS?

I have heard arguments that OSS is cheaper overall, but I don’t really see that cost is the problem (see My first question is WHY? above). It seems to me that the per capita charges for the ePortfolio are pretty reasonable. There is currently no charge made for any supervisor (educational or clinical), programme director, administrator, ARCP panel member or assessor using the ePortfolio. Per capita charges are based only on trainees at present. Would OSS have any impact on this? I can’t see that it would.

Final thoughts

If we were starting from scratch we would look at OSS as one of the options. We would probably look at an off-the-shelf ePortfolio too. We would be foolish not to. But we are not starting from scratch. We have an established, bespoke ePortfolio that is used across the professions (we have versions for Dentists, Nurses & Midwives, Pharmacists, Doctors and Undergraduates), is integrated with a number of external systems and capturing over a million forms submitted by ‘assessors’ every year. Each version has a custom set of features, making it adaptable and cost effective (sharing an underlying code base and database).

Many of the problems we hear about relate to complaints about the educational processes, and changes are already underway to address these (eg move to Supervised Learning Events in Foundation from August 2012). We contribute to these discussions when appropriate.

We have developed good relationships with our broad range of Customers, and continue to work with them to improve our change control and development processes. We work within the constraints of the NHS, which impacts our management of finance, procurement, stakeholder management, technology and decision making, as well as our governance arrangements.

We have an established application and an experienced team, whose expertise and commitment cannot be underestimated. Our development costs are at the lower end of the market, and maintenance charges are extremely good value. We can bring in additional specific expertise as and when we need to.

I can’t help but think the suggestion to move NHS ePortfolio to OSS is a solution to the wrong problem.