Category Archives: WPBA/SLE

News from the trenches

If you have not yet already seen the blog by Dr Fitz “NHS ePortfolio WPBAs in CMT: are they educationally useful?” I recommend you make yourself a cup of tea and have a read. He has taken a significant amount of time out of his day as a medical trainee to document his experiences of using WorkPlace Based Assessments (WPBAs) in the “real world.” He has done this not because he likes to rant, but because he is genuinely interested in how he can support his own learning, and ensure that the assessments he undertakes are valid and useful. He is like many of us – he wants to be a better doctor – but wonders whether the system currently helps or hinders this.

teacup

Dr Fitz seems representative of many Core Medical Trainees (CMTs). In 16 months he has undertaken 51 assessments (ACATs*, CbDs*, miniCEXs*, DOPS* and Teaching Assessments) and 2 rounds of MSFs* (360 degree assessments) with 35 responses from colleagues in all. This is a little more than the minimum requirement for his ARCP (annual appraisal) and is a good bank of data on which to reflect. Well, it should be. Dr Fitz looks in detail at the contents of his ePortfolio and wonders what it really tells him. Of particular concern is the documented feedback that is at the heart of these assessments:

“Unfortunately the stats don’t look good. Over the course of 9 ACATs, covering the management of 55 patients over 12 months, I received 127 words of feedback. That is 14.1 words per ACAT and 2.3 words per patient seen. About 6 tweets.”

He is quick to point out that this does not necessarily reflect the amount or quality of verbal feedback he received, but

“…the educational benefit of my on-calls and the instruction I received from my consultants was separate to, not part of, my ACATs.”

Sadly the story is very similar for his CbDs, miniCexs and DOPS. The quality of the feedback documented on his ePortfolio is poor, and is of no use to him when he reviews his ePortfolio to prepare for his ARCP and consider what should be on his Personal Development Plan (PDP).

His conclusion is balanced and reflects the feelings of many trainees who have commented here previously:

“Overall, my experiences of the NHS ePortfolio assessments for CMT is that whilst they may act as a record of learning, they fail to be a useful educational tool in themselves. This is mainly due to the discord between how they are supposed to be completed and how they are completed in practice. Teaching, supervision and education is happening, but it is in spite of WPBAs rather than because of them.”

enthusiasm for portfoliosHe doesn’t end there and has several suggestions for improvements. They mirror comments already on this blog and range from urgent technical improvements (really, do I have to mention an app yet again?), to faculty development. This is just the kind of input Training Programme Directors, and National Programme leads want and need. Trainees have reasonable and real concerns about their training, and are engaged and enthusiastic about improving it.

Harnessing this enthusiasm will be vital.

If you haven’t already contributed to the Shape of Training review please do so. This is a collaboration between several higher bodies including the General Medical Council, Medical Education England, the Academy of Medical Royal Colleges, the Medical Schools Council, NHS Scotland, NHS Wales and the Conference of Postgraduate Deans of the UK. The review is considering what changes are needed to postgraduate medical training to make sure it continues to meet the needs of patients and health services in the future. This includes options to support greater training and workforce flexibility, and how to address the tensions between obtaining training and providing a service. You have until February to make your voice heard.

* ACAT= acute care assessment tool, CbD = case-based discussion, miniCex = mini Clinical encounter, DOPS = directly observed procedure

Advertisements

Progress

Things have been a little quiet on the site lately, and you would be forgiven for wondering whether I have lost interest in the NHS ePortfolio. You may have started to think that nothing is being achieved, and therefore not bothered to comment or contribute to the discussion.

Don’t believe it!

Progress may be slow, but real change takes time. Quick fixes are great and can have a major impact on functionality. Remember the problems with being unable to link to multiple curriculum items? Fixed! See the tech improvement shopping list for other modifications that have already happened. Quick fixes are also visible and keep up enthusiasm and morale. But they don’t address the route causes of problems and don’t change systems. Changing systems takes time.

A major breakthrough has been the creation of an ePortfolio reference group at the Royal College of Physicians. You can apply to be a member of this panel and get your voice heard directly by the College. Not a member of the RCP? Then ask your own College if they have a user group you can join. And if they don’t have one, ask why not. The systems imposed on trainees are currently not fit for purpose, and we need to make sure people in charge understand this.

Dont forget that NES, the group who run the NHS ePortfolio used by Physicians, Paediatricians and others, is holding feedback events. I’m sure that this is in no small part due to pressure from this site and discussions on Twitter. If you can, go along and make your voice heard.

People are listening.  I have meetings coming up with the Royal College of GPs who use a different ePortfolio system but share common needs. I am also having a follow-up meeting at the Royal College of Physicians. We must clarify the commissioning and costs of the ePortfolio in order to collaborate across Colleges and effect change. When money is scarce we need to make it go further. The Academy of Medical Royal Colleges Trainee Doctor Group are gathering data on the systems in use by all specialities which will be essential is informing this discussion.

It could be so much better! Please keep up your support.

Acronym Soup: RCP, JRCPTB, WPBA, SLE, AoP, AoMRC ATDG

The Royal College of Physicians

This week I met with the Royal College of Physicians in London to discuss the NHS ePortfolio. After the many discussions I have had with trainees and trainers on this blog, on Twitter and in person I felt well prepared. I was sure the RCP should be taking a keen interest in the ePortfolio and it’s place in the wider context of Postgraduate Education, and through some brief email conversations I felt we were likely to have common ground. But as I walked through the falling leaves of Regent’s Park, I wondered what they would make of me and my graffiti-decorated blog…

It would have been unrealistic to expect to leave the meetings with all the answers, but I truly believe we have taken the first step towards a better solution. There are lots of details to work out, and I found myself discussing things like commissioning and procurement – yet another new language to learn! What was so positive was that there was an acknowledgement that trainees (and trainers) are dissatisfied with the ePortfolio, and that the concerns being raised are not idle complaining, but are educationally valid.

This is an important issue because it affects thousands of doctors, and because it relates to other areas of Postgraduate Training. The RCP wants to support trainees in their professional development and acknowledge that the ePortfolio is part of this process. There is also a realisation that technology has moved on, that the ePortfolio was not future-proofed, and that it is important to take stock and think about how to move forward. Financial contraints will impact on these plans and we need to be both realistic and imaginative in our decision making. The fact that the new RCP revalidation tools for Consultants have no interface or link with the trainee ePortfolio is another example of a lack of joined up thinking which we must avoid in future.

I was happy to hear that the RCP have almost finalised the details of the new ePortfolio User Reference group, which will have several trainee representatives. Hopefully this will not only impact on decision making, but also improve communication, so that we don’t in future have changes appear with no warning. It was also encouraging to hear more about the research being done on WPBAs/ SLEs. The RCP are putting considerable time and resources into reviewing the use and utility of WPBAs and the results of this research is likely to have a significant impact on the assessment systems for all medical specialties and shape training for the next 10 years. This is a great opportunity to engage with the College and have a real impact on Postgraduate Training.

There are many questions that do not yet have answers, in particular the question of funding and commissioning ePortfolio systems in the future. All doctors have a core set of common needs and I believe it is essential that we define these together and press for collaboration across Colleges, and open-ness in every part of the process.  The Trainee Roadmap is a first attempt at this, and I encourage you to contribute. The RCP seem to truly be putting trainees’ needs at the heart of what they do, and are (slowly) responding to the concerns of the thousands of you who have visited this site. But the structures of these organisations are complex and my attempts to find a way through the committees and funding streams of the RCP, the JRCPTB and others is already bringing on a headache.

We need to make sure this stays at the top of the agenda for the College, and I look forward to follow-up meetings in the future. I hope the other Colleges are watching, and are considering their own strategies. Otherwise they may find their trainees asking questions they can’t answer….

The Academy of Medical Royal Colleges Trainee Doctor Group

Coincidentally, in the same week I also went to give a presentation at the AoMRC ATDG (seriously, the number of acronyms in the world of medical education is mind-boggling).

There were many nods of assent as I described my frustrations with the current ePortfolio, and the demands placed on trainees and trainers to complete activities with little educational value. The representatives at the ATDG come from a wide spectrum of specialities including O&G, General Surgery, Anaesthetics and Intensive Care, Emergency Medicine, Pathology, General Practice, Ophthalmology, Psychiatry, Radiology and Medicine. Various ePortfolio systems are used by these trainees and some, such as the Haematologists, have to use two systems which, of course, do not talk to each other.

There was also great enthusiasm for sharing information, and for collaboration across Colleges. There was agreement that there are common needs for tools that support learning and professional development, capture workplace learning, log assessments, and provide evidence for appraisal. These needs are shared by doctors of all specialities and span the start of FY1 to retirement (in fact, since four UK medical schools use the NHS ePortfolio, these needs span Undergraduate as well as Postgraduate training). Technical aspects were touched on, and some absolutes were identified: such as a single sign on for all systems; a set of core standards for any ePortfolio used by doctors; and the need for flexibility for Colleges/Specialities and individuals. There was enthusiasm and hope for a future in which there is a simple but flexible ePortfolio system, with mobile support, that truly supports learning, and that makes people smile not scream when they login!

This committee is unique in bringing together trainees across specialities. It’s also full of lovely and enthusiastic people. The representatives are going back to their respective College trainee groups to gather information on what systems are used, and what trainees think of them. I hope we can then finalise a Trainee Roadmap and Core Requirements document, that will help us move forward.

Watch this space!

An app-ortunity

Quite reasonably I have been asked how an NHS ePortfolio app would benefit doctors, and what it would have to do to be worth any investment. In my opinion the need for an app is driven by the need to make WPBAs more relevant. An app would put control back in the hands of trainees, and make life significantly easier for trainers/assessors. This would reduce resentment towards WBPAs and would save an unimaginably huge amount of time for a stressed, squeezed, overworked profession.

The current situation:

I am doctor in training (this covers everyone who is not yet a Consultant/GP partner). I am required to complete a certain number of WPBAs to progress. One day I am at work and am on call admitting new patients to hospital. I think I’ve made a pretty thorough assessment of a patient with a condition I’ve not encountered before and ask my Consultant if, after presentation of the case on the post-take ward round they can fill in a mini-CEX. They say yes.

I present my case during the round and the Consultant provides some useful immediate feedback on my assessment, including a recommendation to read a recent review on the subject in an academic medical journal. However, the Consultant has another 7 patients to review after this and can’t stop to find a computer, login, wait for it to load up, access the NHS ePortfolio website, login and complete the assessment. “Send me a ticket” they say, with a genuine intent to complete is as soon as possible. My shift gets busier and after 13hours at work I go straight to bed when I get home. The next day i am very busy and forget to send the ticket via email. I remember when I get home but realise I don’t know the Consultant’s email address. It’s a weekend so I’m not in for another 2 days. I set a reminder with an alarm on my phone and on Monday the alarm prompts me to retrieve the email  from the hospital system and I send the ticket from the NHS ePortfolio site.

A week later the assessment has not been completed and I send a reminder. Three days after this I bump into the Consultant in the lunch queue and gently remind them about the mini-CEX. They make excuses, feel bad, and promise to do it ASAP.

A week after this the Consultant finally has some time for admin and discovers my reminder email in their inbox. They login and struggle to remember anything about the patient or the feedback they gave me. They have an overall impression of whether I’m any good or not and complete the assessment mainly based on this overall view, rather than the specifics of the case we discussed. I get an email to say that the assessment has been completed. At a later date I login and read the comments, which are brief, and get no educational benefit from the recording of the episode. I do however feel less stressed as that’s one less assessment to get ticked off. I can’t remember the author of the review recommended by the Consultant and never quite get round to searching for it.

A possible future situation:

One day I am at work and am on call admitting new patients to hospital. I think I’ve made a pretty thorough assessment of a patient with a condition I’ve not encountered before and ask my Consultant if, after presentation of the case on the post-take ward round they can fill in a mini-CEX. They say yes.

I present my case during the round and the Consultant provides some useful immediate feedback on my assessment. I get out my smartphone and login to the NHS ePortfolio app. I bring up the mini-CEX form and we complete it together adding comments based on the feedback the Consultant has just given, including the recommendation to read a recent review by author X in journal Y. There is a prompt to enter the Consultant’s email address so that they can validate the mini-CEX as an accurate representation of the assessment, and I input this as the Consultant dictates it. I save the form. The Consultant continues with the post-take ward round. I continue to admit new patients.

When I get home my phone picks up my wifi signal, and the ePortfolio app automatically synchs with my account so that the mini-CEX is uploaded. An email is sent to my Consultant and me to inform us of this new entry on my ePortfolio. I don’t have to waste time chasing up multiple assessments like this, so actually get round to looking up the review recommended by the Consultant, and learn something that will benefit my future patients.

It is essential that an NHS ePortfolio app:

  • is cross platform (iPhone, android etc)
  • can perform most functions offline with synching later with the main site. Most NHS hospitals have no wifi and poor phone sinal coverage. If an app required wifi it would be of no use to many, many, users

Another possible function would be to record reflection-in-action – essentially quick notes about things that happen that are particularly challenging, satisfying etc. There would then be scope to comment on this in the portfolio later (reflection-on-action).  Professionals must be reflective to learn and develop, but there is debate around the value of writing down these reflections. An app would at least make the process easier for those that wished to do this.

Oh, and of course ideally it would be free. But I’d pay £0.69 to make my life easier, wouldn’t you?