Tag Archives: ePortfolio

Tipping point for the NHS ePortfolio?

The ePortfolio is infamous in the world of UK postgraduate training, with few fans and many critics. But where did it come from, and where is it going?  

Where did it come from?

The ePortfolio began as a pilot project, developed by a group within NHS Scotland, a special NHS Health Board. It has expanded beyond anyone’s imagination at the time and now has thousands of users across multiple specialities and multiple stages of training.

Unfortunately the perception of users is that the ePortfolio is unintuitive, time consuming, frustrating and of limited educational value. This frustration is due to views on the competency-based training and assessment system we find ourselves in, poor communication about the reasons behind and timeframe of proposed changes, as well as the software itself.

Where is it going?

A major development has occurred, which has fallen under the radar of most trainees and trainers but potentially has major implications for all. NES have recently published a review paper, in which they acknowledge that the financial and political constraints of being situated in an NHS Health Board mean they cannot do what they need to. They cannot hire the right people for the right time period, or invest in significant improvements to their product. In essence they are unable to function as a technology company and therefore struggle to effectively deliver a technology-based product.

Screen Shot 2013-07-18 at 12.37.45

The review proposes three options:

Option 1:  Do nothing – NES continues to manage and operate ePortfolio directly on a day-to-day basis within the existing management structure and operating constraints.

Option 2:  Further development within NES – To identify and cost the changes required to develop management and financial arrangements within NES to help promote opportunities for the future development of ePortfolio.

Option 3: Joint venture with a third party – NES enters into a joint venture with a third party to transfer operational management of ePortfolio to a third party and retain some level of joint ownership and control over future direction.

Option 1 doesn’t sound great: the status quo is no good. Option 2 might be better but still seems very limited. Option 3 is potentially a major opportunity but includes lots of uncertainty, unknowns and possible risk. And still does not necessarily solve the problem of there being no overview of ALL ePortfolio systems used by ALL doctors and the need for interoperability and future-proofing. The option of an outright was sale is also mentioned but is the less favoured option by NES.

Third parties must register their interest by July 31st, and submit proposals by September 13th. We will then hear the results by October 31st.

All current users should follow developments with interest, and ensure their Royal College makes the right decisions on their behalf. The Royal College of Physicians has a user group and actively seeks the opinions of trainees and supervisors. Other Colleges should follow their lead. Any change potentially has major implications for the thousands of doctors who love to have the ePortfolio.

Back up now!

There is great uncertainty as to the future of the ePortfolio. Considering the importance of the data within it in terms of future job progression and prospects, I would make sure you’ve backed up your data. You’re only option is to download a PDF, but this is better than nothing. And remember to make sure anything in your personal library also exists somewhere else. You never know what the future might hold…

Read the NES Review paper. 

Advertisements

Money makes the world go round

It’s been a while since I blogged, but that doesn’t mean that nothing has been happening. I continue to participate in the RCP ePortfolio group with our next meeting coming up in July so please pass on anything you want me to feedback to them. Also, the teams behind Meducation and Podmedics have joined forces and launched a kickstarter to start developing Oportfolio: a free, open source ePortfolio, designed by doctors for use from medical school to retirement. Is this a game-changer? It could be….

Screen Shot 2013-06-22 at 18.29.12

All doctors need a way to log their achievements and learning. But an ePortfolio should be more than a collection of paperwork mandated for an appraisal or revalidation. It should support an individual in a constant process of self-reflection and improvement, and enhance something that is at the heart of learning – a relationship. The day when trainees and supervisors talk with enthusiasm and joy about their ePortfolio will be a happy one. I can’t wait to see this happen, but looking around at the software available across all specialities and all levels of training it feels a long way off. Why?

Money, money, money

One of the biggest barriers to change is constrained resources. We all know there is not much cash around at the moment in the NHS, and education in particular has suffered. Trainees have complained that the ePortfolio they use is not good value for money, but there is a limited amount that the Colleges (who hold the purse strings) can invest. So how much does the ePortfolio cost?

Photo credit: Images_of_Money http://www.taxbrackets.org/(Creative Commons)

Photo credit: Images_of_Money http://www.taxbrackets.org/(Creative Commons)

In 2012 the NES ePortfolio (the biggest provider) cost it’s customers (Royal Colleges,the Foundation Programme etc) over £600,000. That was just one year. Good software costs money: it is created by talented professionals whose time and expertise is valuable. Putting a number on what good value is for the ePortfolio would be arbitrary  However, if the users think it’s not good value, then it isn’t. They don’t value it, and therefore are less likely to engage with it. Disaster.

Due to the current way in which medical ePortfolios are funded, the only way to make them better, add features and speed up the development of the app would be to pour lots of money in. This is because the funding model depends on the customers (Colleges) individually specifying changes and paying for them in terms of developers’ time. These costs are on top of the day to day running costs. Unless this funding model radically changes, NES will never be able to deliver the kind of ePortfolio doctors are clamouring for. The team at NES have been fantastic in engaging with trainees and encouraging calls for change, but they are constrained by their location in an NHS Scotland Health Board and their lack of power over their own organisational and financial structures.

Don’t be shackled to the past

An important point made by Ben, a NES developer on an open Linked.in thread is that trainees only see one part of structure of the ePortfolio. There is a lot behind the sterile pages of mini-cexs, in terms of levels of access, account types, administration and data reporting. But the counter-question to this is, is that where we should be directing our money? Should we be paying for complicated layers of access structures, or should we trade this for better functionality? It is time for a fundamental shift in our conception of an ePortfolio, putting the learner at the centre. We have the opportunity to re-imagine what an ePortfolio could be and mustn’t let the structure of the current system dominate our thinking.

Open source – what’s all the fuss about?

Some are fearful of open source projects but there is already precedent in the NHS for open source software; for clinicians working directly with software developers to iteratively create things of beauty. If you have not seen OpenEyes before, click and behold! OpenEyes even holds patient data securely, so any alarmist worries about trainee data in an open source system can stop right here. It is fantastic to see new, agile ways of working emerging as successful – and anything has to be better than the NHS’s previous track record on IT projects!

Oportfolio

The Oportfolio team are creating an open source ePortfolio. This means the code will be open for anyone to use. They are not in it for the money. Their motives (paraphrased by me after many conversations with Ed and Jeremy at NHS Hackdays and online) are:

  • a wish to help those who work in the NHS (for the ultimate benefit of patients)
  • an interest in how technology can support education
  • a passion for creating beautiful software.

Oportfolio will available to any doctor in the NHS, including those who are currently not affiliated to a College and therefore have no ePortfolio at all. The potential gains to the NHS are huge in terms of money, and doctor-time. More intangible benefits include: trainee and educator happiness; and more fulfilling postgraduate training.

Ultimately, happier, more reflective, better educated and informed doctors give better patient care. And that’s something you can’t put a price on. 

So whether Oportfolio becomes the best thing since sliced bread and is adopted by all healthcare professionals in the NHS, or whether it is never adopted but creates  pressure for change in the current systems that can no longer be ignored, it will have been a success.

The plan is for the end product to be free to users, but to get started they need a little seed money. I’ve just put my money where my mouth is to help get it off the ground. I hope you do the same.

What can you do?

The RCP are listening – what do you want to tell them?

A few months ago I participated in the first Royal College of Physicians (London) ePortfolio reference group. This group has been established to increase the trainee voice in the ePortfolio development process, and should be celebrated and welcomed. The RCP is leading the way in engaging trainees to make sure their educational experience is as good as possible. Does your College have trainee representation? If not, you should ask them why not…..

Screen Shot 2013-05-20 at 12.29.28

At this meeting we discussed many areas, and tried to keep things positive, focusing on what changes could be made that would make the ePortfolio better for trainees. The RCP agreed to take these issues forward to a meeting with the NHS ePortfolio providers NES:

1. Development of an API to allow integration with third-party tools

It is great that the RCP were willing to take this on and discuss this as a priority with NES. There are lots of apps being developed (including many at NHS Hackdays) which help trainees to capture their learning, for example in log books. There is currently no way that this data can be incorporated into the ePortfolio. Since NES does not have the capacity to create additional software, allowing third party software to interface would be a major step forward in increasing the functionality of the ePortfolio.

2.  HTML5 app to concentrate on work place based assessments

As you may be aware there is now finally an app! Amazing! But at present it can only log reflective practice and send tickets. The group agreed that WPBAs/SLEs should be the priority for the next release of the app.

3. Development of summary views of data to support ARCPs

The current summary overview is unimaginative and unhelpful for ARCPs and PYAs. I have heard too many stories of trainees still having to print their portfolio, or send documents separately. In my own ARCP last year the panel could not find all the evidence they needed and I had to send it as PDFs afterwards. There should be a better way for the trainee to curate the content of the ePortfolio on an ARCP page. It should be possible to define the requirements for ARCP fulfilment and then demonstrate how these have been met. The data should be displayed in an easy to view at-a-glance way so that ARCP panels can rapidly see and access what they need.

4. Change to how ePortfolio roles are created so they are cumulative

Rather than having to switch between roles and select one, with different access levels for each type of access account. This is particularly important for those with significant assessor roles in addition to trainee roles.

Please let us know your thoughts on these issues, and tell us what your priorities are so that we are well prepared for the next meeting, hopefully in June/July. I’m looking forward to hearing how these issues were received and what plans are in place for taking them forward.

We have a real opportunity to raise issues, focus the priorities of those with the purse strings and effect change. Don’t stay silent! 

You can give your feedback in a number of ways:

  1. Comment here (below this post)
  2. Talk to the JRCPTB ePortfolio team
  3. Talk to the your Foundation Programme school
  4. Give feedback to NES directly on Twitter
  5. Give feedback to NES about the app specifically 

Shape of Training – influence the next 30 years of medical training!

You have only days left to shape medical training for the next 30 years.

The Shape of Training review aims to plan how doctors should work and train in the next 30 years. This is your chance to directly tell decision-makers what you want postgraduate training to look like.

Screen Shot 2013-02-04 at 22.49.27

  • Should we have more generalists and fewer specialists?
  • Should there be a speciality of General Internal Medicine (distinct from Geris)?
  • Should all medical trainees CCT in GIM before starting speciality training?
  • Should more specialities dual-accredit and therefore contribute to the acute take (hello Rheumatology, Dermatology, Renal, Oncology……)?
  • Should F2 be abolished?
  • Is training flexible enough?
  • How can trainees be supported to learn from their experiences?  
  • Is the balance right in the current system between training and service provision?

I have strong views on many of these questions (the answer to the last one is NO!)

“This review takes place in a rapidly changing environment. Medical and scientific advances, evolving healthcare and population needs, changes to healthcare systems and professional roles, the push towards more care provided in the community, the information and communications technology (ICT) revolution, and changing patient and public expectations will all affect how doctors will practise in the future. We therefore need to consider what these changes mean for the way doctors are trained.”

The survey is long, but is so important that it’s worth the effort. I recommend you put aside some time, make an extra large cup of tea, and really dedicate some brainpower to your answers. This is the best chance you will ever have of influencing the shape of medical training. Don’t let it slip through your fingers.

In particular I would consider mentioning your views on WPBAs and the ePortfolio in questions

  • 13: How do we make sure doctors in training get the right breadth and quality of learning experiences and time to reflect on these experiences?  (Better software in which reflection could be logged on the go, and reflections could be tagged and organised/visualised/shared more flexibly would help. Time spent with mentors instead of filling in paperwork would also be great)
  • 14: What needs to be done to improve the transitions as doctors move between the different stages of their training and then into independent practice? (Interoperability of ePortfolio systems would be a start)
  • 18: Are there other changes needed to the organisation of medical education and training to make sure it remains fit for purpose in 30 years time that we have not touched on so far in this written call for evidence? (yes…..)

Go on. Respond… 

The perfect ePortfolio

Thanks to @londonlime

Thanks to @londonlime

This weekend I am at NHS Hackday in Oxford. I have written about hackdays before. I am a huge enthusiast. It is amazing to see doctors, other healthcare workers, patients, organisations and software developers coming together in their free time to make stuff that could make the NHS better. A hackday IS agile software development, speeded up, with people motivated not by profit but by intellectual curiosity and a desire to make the world a little bit better.

There are some amazing people coming to NHS Hackday Oxford. Some of them are interested in rethinking a professional ePortfolio. The one we have currently is competent. It does a job. The creators at NES are great, but they are constrained by their history and location. In my opinion an IT project that supports thousands of healthcare professionals’ development should not be run by project managers in a Scottish NHS health board. Their customers are Royal Colleges, not ePortfolio users. However great NES are at their job are they really the best people to make the perfect ePortfolio?

As trainees we want more. In relation to the software we want seamless functionality, we want flexibility, we want personalisation, we want visualisation of data, we want speed, we want interoperability, we want openness and APIs, we want mobile offline data entry, we want intuitive navigation, we want reliability, we want to be encouraged and inspired, we want beauty.

Too much to ask? I hope not. 

I need to form these vague statements into some specifics between now and tomorrow. Wish me luck!

Feedback from NES events

Late last year, NES who run the NHS ePortfolio, held a series of user group meetings. The objectives included “providing an opportunity for users to speak their minds and share their ideas to improve the system.

The results of the feedback events were collated by NES, and they published a summary. So what can we learn from this?

Screen Shot 2013-01-19 at 11.07.28

Firstly we must question whether the results are representative of the users of the ePortfolio. Unfortunately the feedback events were during normal working hours and very few trainees or clinicians were able to attend. It is clear from the feedback that there were a significant number of administrators present. Their views are important but are over-represented in this sample. The sample size of the feedback events is also very small. This is important considering the large number of users and the fact that the use of the system by different Colleges varies.

Despite this there is some useful information contained in the summary, which is grouped into categories:

  • what users like about the NHS ePortfolio (mainly that an ePortfolio is better than paper)
  • dislikes about the technical implementation and website design
  • dislikes related to the educational content or usage requirements of the ePortfolio
  • dislikes relating to the governance of the ePortfolio
  • what technical changes users would like made to the system
  • what new features users would like

The results are not surprising. Many of the comments align with those on this site. Users want an app, want the site to be more intuitive to use and navigate, and want it to have better and more flexible functionality. They want interoperability with other systems and they want clarity on the purpose of the ePortfolio “is it a learning tool or a recording tool?” They also want a lower burden of assessments.

NES must be congratulated for attempting to engage users and gain feedback, and for their openness in releasing the results, but I can’t believe there is anything in the comments that they did not know already. The problem is less lack of information, and more lack of action. But as NES highlights in their post “who makes decisions about the ePortfolio?” in the bigger picture the power really rests with the Colleges. 

Is it time to rethink how we use portfolios and workplace-based assessments in Postgraduate training? How can we adapt the software we have so that it works for us and we maximise the benefits of an ePortfolio for lifelong learning?  

News from the trenches

If you have not yet already seen the blog by Dr Fitz “NHS ePortfolio WPBAs in CMT: are they educationally useful?” I recommend you make yourself a cup of tea and have a read. He has taken a significant amount of time out of his day as a medical trainee to document his experiences of using WorkPlace Based Assessments (WPBAs) in the “real world.” He has done this not because he likes to rant, but because he is genuinely interested in how he can support his own learning, and ensure that the assessments he undertakes are valid and useful. He is like many of us – he wants to be a better doctor – but wonders whether the system currently helps or hinders this.

teacup

Dr Fitz seems representative of many Core Medical Trainees (CMTs). In 16 months he has undertaken 51 assessments (ACATs*, CbDs*, miniCEXs*, DOPS* and Teaching Assessments) and 2 rounds of MSFs* (360 degree assessments) with 35 responses from colleagues in all. This is a little more than the minimum requirement for his ARCP (annual appraisal) and is a good bank of data on which to reflect. Well, it should be. Dr Fitz looks in detail at the contents of his ePortfolio and wonders what it really tells him. Of particular concern is the documented feedback that is at the heart of these assessments:

“Unfortunately the stats don’t look good. Over the course of 9 ACATs, covering the management of 55 patients over 12 months, I received 127 words of feedback. That is 14.1 words per ACAT and 2.3 words per patient seen. About 6 tweets.”

He is quick to point out that this does not necessarily reflect the amount or quality of verbal feedback he received, but

“…the educational benefit of my on-calls and the instruction I received from my consultants was separate to, not part of, my ACATs.”

Sadly the story is very similar for his CbDs, miniCexs and DOPS. The quality of the feedback documented on his ePortfolio is poor, and is of no use to him when he reviews his ePortfolio to prepare for his ARCP and consider what should be on his Personal Development Plan (PDP).

His conclusion is balanced and reflects the feelings of many trainees who have commented here previously:

“Overall, my experiences of the NHS ePortfolio assessments for CMT is that whilst they may act as a record of learning, they fail to be a useful educational tool in themselves. This is mainly due to the discord between how they are supposed to be completed and how they are completed in practice. Teaching, supervision and education is happening, but it is in spite of WPBAs rather than because of them.”

enthusiasm for portfoliosHe doesn’t end there and has several suggestions for improvements. They mirror comments already on this blog and range from urgent technical improvements (really, do I have to mention an app yet again?), to faculty development. This is just the kind of input Training Programme Directors, and National Programme leads want and need. Trainees have reasonable and real concerns about their training, and are engaged and enthusiastic about improving it.

Harnessing this enthusiasm will be vital.

If you haven’t already contributed to the Shape of Training review please do so. This is a collaboration between several higher bodies including the General Medical Council, Medical Education England, the Academy of Medical Royal Colleges, the Medical Schools Council, NHS Scotland, NHS Wales and the Conference of Postgraduate Deans of the UK. The review is considering what changes are needed to postgraduate medical training to make sure it continues to meet the needs of patients and health services in the future. This includes options to support greater training and workforce flexibility, and how to address the tensions between obtaining training and providing a service. You have until February to make your voice heard.

* ACAT= acute care assessment tool, CbD = case-based discussion, miniCex = mini Clinical encounter, DOPS = directly observed procedure