Tag Archives: trainee

Money makes the world go round

It’s been a while since I blogged, but that doesn’t mean that nothing has been happening. I continue to participate in the RCP ePortfolio group with our next meeting coming up in July so please pass on anything you want me to feedback to them. Also, the teams behind Meducation and Podmedics have joined forces and launched a kickstarter to start developing Oportfolio: a free, open source ePortfolio, designed by doctors for use from medical school to retirement. Is this a game-changer? It could be….

Screen Shot 2013-06-22 at 18.29.12

All doctors need a way to log their achievements and learning. But an ePortfolio should be more than a collection of paperwork mandated for an appraisal or revalidation. It should support an individual in a constant process of self-reflection and improvement, and enhance something that is at the heart of learning – a relationship. The day when trainees and supervisors talk with enthusiasm and joy about their ePortfolio will be a happy one. I can’t wait to see this happen, but looking around at the software available across all specialities and all levels of training it feels a long way off. Why?

Money, money, money

One of the biggest barriers to change is constrained resources. We all know there is not much cash around at the moment in the NHS, and education in particular has suffered. Trainees have complained that the ePortfolio they use is not good value for money, but there is a limited amount that the Colleges (who hold the purse strings) can invest. So how much does the ePortfolio cost?

Photo credit: Images_of_Money http://www.taxbrackets.org/(Creative Commons)

Photo credit: Images_of_Money http://www.taxbrackets.org/(Creative Commons)

In 2012 the NES ePortfolio (the biggest provider) cost it’s customers (Royal Colleges,the Foundation Programme etc) over £600,000. That was just one year. Good software costs money: it is created by talented professionals whose time and expertise is valuable. Putting a number on what good value is for the ePortfolio would be arbitrary  However, if the users think it’s not good value, then it isn’t. They don’t value it, and therefore are less likely to engage with it. Disaster.

Due to the current way in which medical ePortfolios are funded, the only way to make them better, add features and speed up the development of the app would be to pour lots of money in. This is because the funding model depends on the customers (Colleges) individually specifying changes and paying for them in terms of developers’ time. These costs are on top of the day to day running costs. Unless this funding model radically changes, NES will never be able to deliver the kind of ePortfolio doctors are clamouring for. The team at NES have been fantastic in engaging with trainees and encouraging calls for change, but they are constrained by their location in an NHS Scotland Health Board and their lack of power over their own organisational and financial structures.

Don’t be shackled to the past

An important point made by Ben, a NES developer on an open Linked.in thread is that trainees only see one part of structure of the ePortfolio. There is a lot behind the sterile pages of mini-cexs, in terms of levels of access, account types, administration and data reporting. But the counter-question to this is, is that where we should be directing our money? Should we be paying for complicated layers of access structures, or should we trade this for better functionality? It is time for a fundamental shift in our conception of an ePortfolio, putting the learner at the centre. We have the opportunity to re-imagine what an ePortfolio could be and mustn’t let the structure of the current system dominate our thinking.

Open source – what’s all the fuss about?

Some are fearful of open source projects but there is already precedent in the NHS for open source software; for clinicians working directly with software developers to iteratively create things of beauty. If you have not seen OpenEyes before, click and behold! OpenEyes even holds patient data securely, so any alarmist worries about trainee data in an open source system can stop right here. It is fantastic to see new, agile ways of working emerging as successful – and anything has to be better than the NHS’s previous track record on IT projects!

Oportfolio

The Oportfolio team are creating an open source ePortfolio. This means the code will be open for anyone to use. They are not in it for the money. Their motives (paraphrased by me after many conversations with Ed and Jeremy at NHS Hackdays and online) are:

  • a wish to help those who work in the NHS (for the ultimate benefit of patients)
  • an interest in how technology can support education
  • a passion for creating beautiful software.

Oportfolio will available to any doctor in the NHS, including those who are currently not affiliated to a College and therefore have no ePortfolio at all. The potential gains to the NHS are huge in terms of money, and doctor-time. More intangible benefits include: trainee and educator happiness; and more fulfilling postgraduate training.

Ultimately, happier, more reflective, better educated and informed doctors give better patient care. And that’s something you can’t put a price on. 

So whether Oportfolio becomes the best thing since sliced bread and is adopted by all healthcare professionals in the NHS, or whether it is never adopted but creates  pressure for change in the current systems that can no longer be ignored, it will have been a success.

The plan is for the end product to be free to users, but to get started they need a little seed money. I’ve just put my money where my mouth is to help get it off the ground. I hope you do the same.

What can you do?

NHS Hackday and the oPortfol.io

This weekend I was at NHS Hackday. Doctors, other healthcare workers, students, patients, organisations and software developers came together in their free time to make stuff that could make the NHS better.

Thanks to @londonlime

Thanks to @londonlime

I was astounded by the last Hackday. I didn’t think my expectations could be surpassed. They were.

The projects were diverse, aiming to solve everyday problems at all levels of the NHS. You can see the list on the Google doc and they’ll be on the wiki soon. The highly deserved winner was OpenHeart. The team used the amazing open source electronic health record at Moorfields Hospital, Open Eyes, and adapted it for use in Cardiology. The end result was stunning. It will save hours of doctors’ time, will create patient records that are much more understandable for patients themselves and for GPs, and will improve communication and therefore the quality of care.

Another favourite was Dementia scrapbook, an app to allow family and friends to contribute to a virtual scrapbook of memories and reminders. It has an easy to use touch interface that can be used by carers or people with dementia themselves. Dementia is common and this takes a very patient-centred approach to solving problems many of us may face in the future. I hope to see it available soon on the app store.

Cellcountr, initially hacked at the Liverpool hackday, was built on with additional features such as data visualisation and a customisable keyboard. It will be launched in the next month at a Pathology conference, and will then make a real difference to doctors, and their ability to accurately diagnosis patients with haematological conditions.

So what did we do? We created oPortfolio, an open API which allows trainee doctors to record learning events online, offline and on the go. It includes a webapp, a mobile-friendly site, an iPhone app, and an android app that all synch data. From nothing to all this in 36 hours! The team were incredible: full of talent, patience, and creativity.

Oportfolio

What does it do?

It solves an immediate need to log learning events on the go (see examples below). It lays the foundation for a more complex system to log assessments and meetings. With (quite a bit) more work it could be a simple open portfolio that doctors who are not currently in a training programme (eg LATs, people doing fellow jobs in between F2 and speciality training) could use to track their professional development. The funding model would have to be clarified as development and hosting is not free! It could also be an arena to experiment with and showcase new ways of organising a professional portfolio that could usefully feed into the debate on what and who a portfolio is for. It could highlight how different systems talking to each other and 3rd party apps and plug-ins have the potential to improve a core product. Another fabulous creation was Quicklog, an app to log personal development in performing procedures on the go. They built in data visualisation to encourage reflection and chart progress. It would be fantastic if the data from Quicklog could be integrated into a portfolio system. Anyone who is interested (and understands it!)  should look at the code on github for oPortfolio and Quicklog!

Screen Shot 2013-01-28 at 11.55.34

What does it not do?

It is not a replacement for the current ePortfolio system(s). The NES ePortfolio and others (eg surgical portfolio) are complex structures build up over years, with thousands and thousands of pounds of investment. Many have questioned whether they are value for money and I can’t answer that but good software does cost. Existing systems have  layers of access rights and methods of data extraction since these were priorities for the bodies who paid for them. They have cloud hosting and data security. People have spent years making them do what they do and it would be crazy to think they could be replicated in a weekend. They have their problems and must be improved but they are here to stay until a better alternative exists.

The focus of building a model Oportfolio was the user experience. If it was developed further it could fulfil a need for trainees who are not in a current training programme, who currently use various cobbled together documents on Evernote, phone notes apps, word documents and paper to record their learning and showcase their achievements when applying for jobs. With regular end-user input it could be beautiful, and a joy to use!

I am sure that our exploits this weekend will appear highly challenging and controversial to some. But I am not controversial. I have always highlighted the frustrations felt by trainees (which are well known) but advocated for engagement with all interested parties: individual trainees, trainers/educational supervisors, LETBs, Trusts, Royal Colleges, current ePortfolio provider NES, the GMC and HEE. We need to get our heads together and think about what the future of training will look like, what tools are needed to enhance learning, and how they will be funded.

The NHS can’t keep putting up with unintuitive, inflexible IT that doesn’t match the realities of practice. As demonstrated at NHS Hackday; intelligence, enthusiasm, creativity, a few humous sandwiches and some coffee can create magic. But that magic needs support and investment to make it sustainable. Muir Gray says change in the NHS will come from the bottom up. He is one of a few inspirational people at the top supporting projects in which frontline staff make a difference. We could do with a few more like him….

The perfect ePortfolio

Thanks to @londonlime

Thanks to @londonlime

This weekend I am at NHS Hackday in Oxford. I have written about hackdays before. I am a huge enthusiast. It is amazing to see doctors, other healthcare workers, patients, organisations and software developers coming together in their free time to make stuff that could make the NHS better. A hackday IS agile software development, speeded up, with people motivated not by profit but by intellectual curiosity and a desire to make the world a little bit better.

There are some amazing people coming to NHS Hackday Oxford. Some of them are interested in rethinking a professional ePortfolio. The one we have currently is competent. It does a job. The creators at NES are great, but they are constrained by their history and location. In my opinion an IT project that supports thousands of healthcare professionals’ development should not be run by project managers in a Scottish NHS health board. Their customers are Royal Colleges, not ePortfolio users. However great NES are at their job are they really the best people to make the perfect ePortfolio?

As trainees we want more. In relation to the software we want seamless functionality, we want flexibility, we want personalisation, we want visualisation of data, we want speed, we want interoperability, we want openness and APIs, we want mobile offline data entry, we want intuitive navigation, we want reliability, we want to be encouraged and inspired, we want beauty.

Too much to ask? I hope not. 

I need to form these vague statements into some specifics between now and tomorrow. Wish me luck!

News from the trenches

If you have not yet already seen the blog by Dr Fitz “NHS ePortfolio WPBAs in CMT: are they educationally useful?” I recommend you make yourself a cup of tea and have a read. He has taken a significant amount of time out of his day as a medical trainee to document his experiences of using WorkPlace Based Assessments (WPBAs) in the “real world.” He has done this not because he likes to rant, but because he is genuinely interested in how he can support his own learning, and ensure that the assessments he undertakes are valid and useful. He is like many of us – he wants to be a better doctor – but wonders whether the system currently helps or hinders this.

teacup

Dr Fitz seems representative of many Core Medical Trainees (CMTs). In 16 months he has undertaken 51 assessments (ACATs*, CbDs*, miniCEXs*, DOPS* and Teaching Assessments) and 2 rounds of MSFs* (360 degree assessments) with 35 responses from colleagues in all. This is a little more than the minimum requirement for his ARCP (annual appraisal) and is a good bank of data on which to reflect. Well, it should be. Dr Fitz looks in detail at the contents of his ePortfolio and wonders what it really tells him. Of particular concern is the documented feedback that is at the heart of these assessments:

“Unfortunately the stats don’t look good. Over the course of 9 ACATs, covering the management of 55 patients over 12 months, I received 127 words of feedback. That is 14.1 words per ACAT and 2.3 words per patient seen. About 6 tweets.”

He is quick to point out that this does not necessarily reflect the amount or quality of verbal feedback he received, but

“…the educational benefit of my on-calls and the instruction I received from my consultants was separate to, not part of, my ACATs.”

Sadly the story is very similar for his CbDs, miniCexs and DOPS. The quality of the feedback documented on his ePortfolio is poor, and is of no use to him when he reviews his ePortfolio to prepare for his ARCP and consider what should be on his Personal Development Plan (PDP).

His conclusion is balanced and reflects the feelings of many trainees who have commented here previously:

“Overall, my experiences of the NHS ePortfolio assessments for CMT is that whilst they may act as a record of learning, they fail to be a useful educational tool in themselves. This is mainly due to the discord between how they are supposed to be completed and how they are completed in practice. Teaching, supervision and education is happening, but it is in spite of WPBAs rather than because of them.”

enthusiasm for portfoliosHe doesn’t end there and has several suggestions for improvements. They mirror comments already on this blog and range from urgent technical improvements (really, do I have to mention an app yet again?), to faculty development. This is just the kind of input Training Programme Directors, and National Programme leads want and need. Trainees have reasonable and real concerns about their training, and are engaged and enthusiastic about improving it.

Harnessing this enthusiasm will be vital.

If you haven’t already contributed to the Shape of Training review please do so. This is a collaboration between several higher bodies including the General Medical Council, Medical Education England, the Academy of Medical Royal Colleges, the Medical Schools Council, NHS Scotland, NHS Wales and the Conference of Postgraduate Deans of the UK. The review is considering what changes are needed to postgraduate medical training to make sure it continues to meet the needs of patients and health services in the future. This includes options to support greater training and workforce flexibility, and how to address the tensions between obtaining training and providing a service. You have until February to make your voice heard.

* ACAT= acute care assessment tool, CbD = case-based discussion, miniCex = mini Clinical encounter, DOPS = directly observed procedure

Progress

Things have been a little quiet on the site lately, and you would be forgiven for wondering whether I have lost interest in the NHS ePortfolio. You may have started to think that nothing is being achieved, and therefore not bothered to comment or contribute to the discussion.

Don’t believe it!

Progress may be slow, but real change takes time. Quick fixes are great and can have a major impact on functionality. Remember the problems with being unable to link to multiple curriculum items? Fixed! See the tech improvement shopping list for other modifications that have already happened. Quick fixes are also visible and keep up enthusiasm and morale. But they don’t address the route causes of problems and don’t change systems. Changing systems takes time.

A major breakthrough has been the creation of an ePortfolio reference group at the Royal College of Physicians. You can apply to be a member of this panel and get your voice heard directly by the College. Not a member of the RCP? Then ask your own College if they have a user group you can join. And if they don’t have one, ask why not. The systems imposed on trainees are currently not fit for purpose, and we need to make sure people in charge understand this.

Dont forget that NES, the group who run the NHS ePortfolio used by Physicians, Paediatricians and others, is holding feedback events. I’m sure that this is in no small part due to pressure from this site and discussions on Twitter. If you can, go along and make your voice heard.

People are listening.  I have meetings coming up with the Royal College of GPs who use a different ePortfolio system but share common needs. I am also having a follow-up meeting at the Royal College of Physicians. We must clarify the commissioning and costs of the ePortfolio in order to collaborate across Colleges and effect change. When money is scarce we need to make it go further. The Academy of Medical Royal Colleges Trainee Doctor Group are gathering data on the systems in use by all specialities which will be essential is informing this discussion.

It could be so much better! Please keep up your support.

Acronym Soup: RCP, JRCPTB, WPBA, SLE, AoP, AoMRC ATDG

The Royal College of Physicians

This week I met with the Royal College of Physicians in London to discuss the NHS ePortfolio. After the many discussions I have had with trainees and trainers on this blog, on Twitter and in person I felt well prepared. I was sure the RCP should be taking a keen interest in the ePortfolio and it’s place in the wider context of Postgraduate Education, and through some brief email conversations I felt we were likely to have common ground. But as I walked through the falling leaves of Regent’s Park, I wondered what they would make of me and my graffiti-decorated blog…

It would have been unrealistic to expect to leave the meetings with all the answers, but I truly believe we have taken the first step towards a better solution. There are lots of details to work out, and I found myself discussing things like commissioning and procurement – yet another new language to learn! What was so positive was that there was an acknowledgement that trainees (and trainers) are dissatisfied with the ePortfolio, and that the concerns being raised are not idle complaining, but are educationally valid.

This is an important issue because it affects thousands of doctors, and because it relates to other areas of Postgraduate Training. The RCP wants to support trainees in their professional development and acknowledge that the ePortfolio is part of this process. There is also a realisation that technology has moved on, that the ePortfolio was not future-proofed, and that it is important to take stock and think about how to move forward. Financial contraints will impact on these plans and we need to be both realistic and imaginative in our decision making. The fact that the new RCP revalidation tools for Consultants have no interface or link with the trainee ePortfolio is another example of a lack of joined up thinking which we must avoid in future.

I was happy to hear that the RCP have almost finalised the details of the new ePortfolio User Reference group, which will have several trainee representatives. Hopefully this will not only impact on decision making, but also improve communication, so that we don’t in future have changes appear with no warning. It was also encouraging to hear more about the research being done on WPBAs/ SLEs. The RCP are putting considerable time and resources into reviewing the use and utility of WPBAs and the results of this research is likely to have a significant impact on the assessment systems for all medical specialties and shape training for the next 10 years. This is a great opportunity to engage with the College and have a real impact on Postgraduate Training.

There are many questions that do not yet have answers, in particular the question of funding and commissioning ePortfolio systems in the future. All doctors have a core set of common needs and I believe it is essential that we define these together and press for collaboration across Colleges, and open-ness in every part of the process.  The Trainee Roadmap is a first attempt at this, and I encourage you to contribute. The RCP seem to truly be putting trainees’ needs at the heart of what they do, and are (slowly) responding to the concerns of the thousands of you who have visited this site. But the structures of these organisations are complex and my attempts to find a way through the committees and funding streams of the RCP, the JRCPTB and others is already bringing on a headache.

We need to make sure this stays at the top of the agenda for the College, and I look forward to follow-up meetings in the future. I hope the other Colleges are watching, and are considering their own strategies. Otherwise they may find their trainees asking questions they can’t answer….

The Academy of Medical Royal Colleges Trainee Doctor Group

Coincidentally, in the same week I also went to give a presentation at the AoMRC ATDG (seriously, the number of acronyms in the world of medical education is mind-boggling).

There were many nods of assent as I described my frustrations with the current ePortfolio, and the demands placed on trainees and trainers to complete activities with little educational value. The representatives at the ATDG come from a wide spectrum of specialities including O&G, General Surgery, Anaesthetics and Intensive Care, Emergency Medicine, Pathology, General Practice, Ophthalmology, Psychiatry, Radiology and Medicine. Various ePortfolio systems are used by these trainees and some, such as the Haematologists, have to use two systems which, of course, do not talk to each other.

There was also great enthusiasm for sharing information, and for collaboration across Colleges. There was agreement that there are common needs for tools that support learning and professional development, capture workplace learning, log assessments, and provide evidence for appraisal. These needs are shared by doctors of all specialities and span the start of FY1 to retirement (in fact, since four UK medical schools use the NHS ePortfolio, these needs span Undergraduate as well as Postgraduate training). Technical aspects were touched on, and some absolutes were identified: such as a single sign on for all systems; a set of core standards for any ePortfolio used by doctors; and the need for flexibility for Colleges/Specialities and individuals. There was enthusiasm and hope for a future in which there is a simple but flexible ePortfolio system, with mobile support, that truly supports learning, and that makes people smile not scream when they login!

This committee is unique in bringing together trainees across specialities. It’s also full of lovely and enthusiastic people. The representatives are going back to their respective College trainee groups to gather information on what systems are used, and what trainees think of them. I hope we can then finalise a Trainee Roadmap and Core Requirements document, that will help us move forward.

Watch this space!

What would you ask?

As a result of this blog, and “a little help from my friends” I have a meeting next week at the Royal College of Physicians to discuss the NHS ePortfolio. I hope to articulate the discussions on this blog, and those that I have had over email, twitter and in person with many of you.

I hope to communicate the frustration felt by trainees and others about the NHS ePortfolio, along with the hope that it could be something so much better: a true educational and professional development tool to support lifelong learning. I hope to gain a better understanding of the views of the Colleges, in particular the view of the governance structures, the funding streams, and the future vision.

Broad areas I hope to discuss include:

  • lack of user input on any decisions about the ePortfolio and proposals to change this
  • poor communication of the rationale for, and timing of changes (personally, I had no warning about the change to formative and summative DOPS) and the effect this has on demoralised trainees
  • technical limitations – such as speed, which increase frustration and decrease utility
  • design limitations – such as lack of flexibility, poor organisation and visualisation of content so that it is hidden away and can’t effectively inform PDPs. These design limitations have an adverse effect on the educational utility of the ePortfolio
  • educational limitations due to lack of an app, meaning delays between a workplace-based assessment being observed, and completed online (therefore feedback becomes meaningless)
  • other limitations – lack of ability to share content with anyone other than supervisors, lack of interoperability and exportability of data for more creative uses
  • what does the College believe is the answer to the question “who owns the data in the ePortfolio?”
  • is there scope for greater collaboration between JRCPTB, RCPCH, The Foundation Programme and other groups who commission their ePortfolios through NES, to invest jointly for change – an app would be a perfect opportunity for this. The users of these systems share many basic needs, and could have individual specifics built in
  • what is the future of the ePortfolio, in technical, and educational terms, and how will this be decided?
  • how is the ePortfolio funded and commissioned? What are the plans for the future? How can we improve the process and increase transparency?
  • the concept of an ePortfolio has great potential but is not delivering what is needed. Investment is essential. User input is essential. There is a great opportunity to get this right and improve both the perception and reality of postgraduate training. Who will lead this change? Will it be the RCP?

I am very grateful to the RCP for taking the time to meet with me, and hope it will be a very constructive and positive conversation. Please let me know what else you feel is essential for me to communicate, as we begin to get closer to a process for change.

What would you ask? Please let me know by commenting below. You can also still edit the ePortfolio Trainee Roadmap