Tag Archives: collaboration

The perfect ePortfolio

Thanks to @londonlime

Thanks to @londonlime

This weekend I am at NHS Hackday in Oxford. I have written about hackdays before. I am a huge enthusiast. It is amazing to see doctors, other healthcare workers, patients, organisations and software developers coming together in their free time to make stuff that could make the NHS better. A hackday IS agile software development, speeded up, with people motivated not by profit but by intellectual curiosity and a desire to make the world a little bit better.

There are some amazing people coming to NHS Hackday Oxford. Some of them are interested in rethinking a professional ePortfolio. The one we have currently is competent. It does a job. The creators at NES are great, but they are constrained by their history and location. In my opinion an IT project that supports thousands of healthcare professionals’ development should not be run by project managers in a Scottish NHS health board. Their customers are Royal Colleges, not ePortfolio users. However great NES are at their job are they really the best people to make the perfect ePortfolio?

As trainees we want more. In relation to the software we want seamless functionality, we want flexibility, we want personalisation, we want visualisation of data, we want speed, we want interoperability, we want openness and APIs, we want mobile offline data entry, we want intuitive navigation, we want reliability, we want to be encouraged and inspired, we want beauty.

Too much to ask? I hope not. 

I need to form these vague statements into some specifics between now and tomorrow. Wish me luck!

Advertisements

Acronym Soup: RCP, JRCPTB, WPBA, SLE, AoP, AoMRC ATDG

The Royal College of Physicians

This week I met with the Royal College of Physicians in London to discuss the NHS ePortfolio. After the many discussions I have had with trainees and trainers on this blog, on Twitter and in person I felt well prepared. I was sure the RCP should be taking a keen interest in the ePortfolio and it’s place in the wider context of Postgraduate Education, and through some brief email conversations I felt we were likely to have common ground. But as I walked through the falling leaves of Regent’s Park, I wondered what they would make of me and my graffiti-decorated blog…

It would have been unrealistic to expect to leave the meetings with all the answers, but I truly believe we have taken the first step towards a better solution. There are lots of details to work out, and I found myself discussing things like commissioning and procurement – yet another new language to learn! What was so positive was that there was an acknowledgement that trainees (and trainers) are dissatisfied with the ePortfolio, and that the concerns being raised are not idle complaining, but are educationally valid.

This is an important issue because it affects thousands of doctors, and because it relates to other areas of Postgraduate Training. The RCP wants to support trainees in their professional development and acknowledge that the ePortfolio is part of this process. There is also a realisation that technology has moved on, that the ePortfolio was not future-proofed, and that it is important to take stock and think about how to move forward. Financial contraints will impact on these plans and we need to be both realistic and imaginative in our decision making. The fact that the new RCP revalidation tools for Consultants have no interface or link with the trainee ePortfolio is another example of a lack of joined up thinking which we must avoid in future.

I was happy to hear that the RCP have almost finalised the details of the new ePortfolio User Reference group, which will have several trainee representatives. Hopefully this will not only impact on decision making, but also improve communication, so that we don’t in future have changes appear with no warning. It was also encouraging to hear more about the research being done on WPBAs/ SLEs. The RCP are putting considerable time and resources into reviewing the use and utility of WPBAs and the results of this research is likely to have a significant impact on the assessment systems for all medical specialties and shape training for the next 10 years. This is a great opportunity to engage with the College and have a real impact on Postgraduate Training.

There are many questions that do not yet have answers, in particular the question of funding and commissioning ePortfolio systems in the future. All doctors have a core set of common needs and I believe it is essential that we define these together and press for collaboration across Colleges, and open-ness in every part of the process.  The Trainee Roadmap is a first attempt at this, and I encourage you to contribute. The RCP seem to truly be putting trainees’ needs at the heart of what they do, and are (slowly) responding to the concerns of the thousands of you who have visited this site. But the structures of these organisations are complex and my attempts to find a way through the committees and funding streams of the RCP, the JRCPTB and others is already bringing on a headache.

We need to make sure this stays at the top of the agenda for the College, and I look forward to follow-up meetings in the future. I hope the other Colleges are watching, and are considering their own strategies. Otherwise they may find their trainees asking questions they can’t answer….

The Academy of Medical Royal Colleges Trainee Doctor Group

Coincidentally, in the same week I also went to give a presentation at the AoMRC ATDG (seriously, the number of acronyms in the world of medical education is mind-boggling).

There were many nods of assent as I described my frustrations with the current ePortfolio, and the demands placed on trainees and trainers to complete activities with little educational value. The representatives at the ATDG come from a wide spectrum of specialities including O&G, General Surgery, Anaesthetics and Intensive Care, Emergency Medicine, Pathology, General Practice, Ophthalmology, Psychiatry, Radiology and Medicine. Various ePortfolio systems are used by these trainees and some, such as the Haematologists, have to use two systems which, of course, do not talk to each other.

There was also great enthusiasm for sharing information, and for collaboration across Colleges. There was agreement that there are common needs for tools that support learning and professional development, capture workplace learning, log assessments, and provide evidence for appraisal. These needs are shared by doctors of all specialities and span the start of FY1 to retirement (in fact, since four UK medical schools use the NHS ePortfolio, these needs span Undergraduate as well as Postgraduate training). Technical aspects were touched on, and some absolutes were identified: such as a single sign on for all systems; a set of core standards for any ePortfolio used by doctors; and the need for flexibility for Colleges/Specialities and individuals. There was enthusiasm and hope for a future in which there is a simple but flexible ePortfolio system, with mobile support, that truly supports learning, and that makes people smile not scream when they login!

This committee is unique in bringing together trainees across specialities. It’s also full of lovely and enthusiastic people. The representatives are going back to their respective College trainee groups to gather information on what systems are used, and what trainees think of them. I hope we can then finalise a Trainee Roadmap and Core Requirements document, that will help us move forward.

Watch this space!

What to do with all that data?

I encourage you all to read the blog of NHS ePortfolio developer @zingmatter, and look at the presentation he presented at AMEE 2012 (a recent Medical Education Conference) “Assessing NHS ePortfolio behaviour: variations in the online activity of doctors as they progress through training.”

With thanks to http://www.acunu.com/ for the badge!

I was at the presentation at AMEE 2012 and, although the presentation title may nor sound gripping, I was fascinated to hear what could be learnt from the vast amounts of data ready and waiting to be analysed on NHS ePortfolio site use. The development team (including @zingmatter) had done a great job of drilling down into some of the data, using Google Analytics and internal tracking,  in order to filter out some meaningful information from the thousands and thousands of logins and episodes over a year.

The Prezi can be seen in the “elastic elephant” blog

However, my first thoughts on seeing the conclusions of the presentation were “they’re asking the wrong questions” and “if they wanted to know that they should have just asked the trainees.” Many of the peaks and troughs seen on the graphs were entirely predictable (ie pre-ARCP), and some of the conclusions drawn by the developers on “depth of use” were weak. I could explain away many of the findings, as I know how trainees use the site is a function of what hoops are put in front of them to jump through. I was also sceptical about the conclusion that trainees change their behaviour in relation to the ePortfolio over the course of their training. FYs and ST6s may interact differently with the site, but there are so many confounders that a snapshot comparison is not a valid way to assess this: a longitudinal study would be required.

Despite these reservations, reading @zingmatter’s blog gives me hope for the future, as the developers at NES are committed to engaging with the needs of users. In our often passionate discussions on social media (including this blog and twitter) we must remember that we come from very different perspectives, and have unique sets of knowledge and skills.

As @zingmatter points out:

“There is a balance between college needs and trainee needs in the design of an e-portfolio and possibly this type of data can help inform this debate.”

We also have to make sure we are not misdirecting our frustration at the wrong people, and potentially alienating them:

“while I’m happy to ask simple questions about user flow, user experience and so on, questions about the educational implications of this data have not been well addressed as it’s not really in my sphere of knowledge (or in my job description). I would see the research I presented at this conference as a ‘this is the kind of thing we can do’ exercise that should lead on to better designed questions that will allow us to understand how best to develop an e-portfolio that supports effective learning and development through the effective delivery of a training programme.”

I really hope we can work together to ask the right questions and use all the data we have to inform the process. All we need now is the Royal Colleges on board and we can really maximise the potential of the ePortfolio.

Just imagine a world in which trainees didn’t hate the NHS ePortfolio. It has the potential to be a useful tool to encourage self-directed learning, provide evidence of experience and achievements, act as a showcase for job applications and excellence awards, and strengthen the relationship between trainee and trainer. This world is far away, but perhaps we are starting to see the path forward…