The NHS ePortfolio fan club

A recent twitter post directed me to the blog of Dr Fiona Pathiraja, which I encourage you to read. She asks:

Am I the only member of the ePortfolio fan club? If the recent vitriol on Twitter is anything to go by, one would be forgiven for thinking that the fan club comprises n=1.

She suggests that much of the negative press is unfair, and that medics don’t know how lucky we are, especially when compared to the private sector.

However, looking more closely her comments are a mix of praise for the concept of an ePortfolio itself, the structured nature of medical training programmes, and the job security that so many others rightly envy. She is honest about the shortcomings of the NHS ePortfolio in its current form and the need for change.

Of course, the ePortfolio is far from perfect. The end user is typically generation Y and expects technology to have the beautiful aesthetics and seamless functionality of their i-products. Improving functionality e.g. linking curriculum items to assessments, and aesthetics is essential for an improved ePortfolio.

This is exactly the point. We expect more. We deserve more. It is time for investment in the NHS ePortfolio to bring its functionality and aesthetics into the era of generation Y.

She also goes beyond the technical aspects of the current ePortfolio and makes requests for: a better and more transparent evidence-base for assessment tools including WPBAs; more training for supervisors on use of assessments; and the need to treat medics as adult learners:

As a cohort of intelligent, motivated young people, we are able to take responsibility for adult learning, but need to be treated like adults in order to do so.  If the ePortfolio gods are listening—engage with us, take constructive feedback, and try to improve the portfolio to inspire the medical generation Y.

I couldn’t have said it better myself. This site is a first step in connecting users, developers, and Royal Colleges.  If we can all work together to effect these changes, I might just join the fan club myself!

Advertisements

13 responses to “The NHS ePortfolio fan club

  1. The more voices the better! I think as you point out, experience of portfolios to assess competency seems to unite rather than divide people. I think that you are providing excellent leadership by trying to engage people around a common purpose of making things better. Too often when things are not going well the tendency is to point to others and ask them to sort it out. But here you are saying that it is up to all of us to get involved and start thinking about the way forwards. I have seen the engagement by the NES developers and have nothing but praise for their engagement too. Let’s hope that all the stakeholders get involved.

    This kind of public discourse about models of education is the future. We all have to learn how to make it work for all concerned. Well done!

  2. Nice to see a different perspective. With such a large number of trainees, supervisors and administrators using the ePortfolio it’s bound to generate a diverse range of views.

    When it started there was a deliberate intention to avoid slick graphics which at that time would have a) diverted resources away from getting the underlying technology set up and b) would have made it unusable in many NHS premises.

    Having said that, time and technology moves on and we are looking to improve the site and bring it up to date. We do still have to bear in mind the NHS IT infrastructure and some things may not be possible because of this.

    We also have a broad ranging group of stakeholders who all use and contribute to this shared system, and I hope we can continue to approach changes in an open and collaborative way.

    It’s very good for the ePortfolio team to hear these views, so thanks to all who have contributed so far.

  3. Thanks for the blog post on my BMJ blog and great post.

    As I said in my post, I think that whilst the ePortfolio has its shortcomings and frustrations, I do think that overall, it adds value to a medical career. This is based on my experiences in the private sector and civil service. I wrote the blog to present a more positive view to the debate on the ePortfolio as there are several vociferous views on how terrible it is. There have been literally hundreds of tweets on this by a vast range of people.

    The problem with Twitter (in my opinion) is that sometimes it is only the people who tweet the loudest/the most/have most followers who get heard. Some may just be looking for a debate for the sake of it or lack organisational influence so try to leverage a perceived influence via social media. If I met them at a conference, I am doubt I would find them credible in real life. In short, I felt there is a skewed opinion on the ePortfolio which may lead the AOMRC to incorrectly believe that this is the global view, hence my post.

    I hope the AOMRC will seek to improve and update the ePortfolio to inspire trainees. I look forward to you joining the fan club.

    FP

    • Fiona,

      You raised many of the same issues as several other people who have commented on Twitter and elsewhere.
      I think that LJ is in the fanclub already- in the sense that she is working to articulate a positive response to a difficult situation.

      It would be really useful if you could explain more about how the postfolio is working for you.

      AM

  4. Dr Pathiraja has asked me to expand on my acerbic 140 character rebuff of the ePortfolio system.
    At work I am a smiling, carefree nonchalant chap – albeit one who harbours a deep, dark secret. I resent the eportfolio; in fact I hate it.

    Time consuming
    The benefits of an electronic portfolio are that it is systematic, generic and permanent. Therein lies the eportfolio’s flaws – in that it is time-consuming, inflexible and not dynamic. I begrudge the cost of this static form; paying over seven hundred pounds for something which does not specifically reflect me or my training needs.
    The eportfolio system is a thief of time – stealing hours from anxious, overworked juniors when morale is already at a low ebb. Each ‘doctor hour’ spent on creating ‘a personal development plan’ or on forced ‘reflection’ can be multiplied across the nation. Taken as a whole these are tens of thousands of valuable ‘doctor hours’ lost – creating prose that is never read; this theft is criminal.
    The phenomena of the ‘rush’ to have noddy Likert forms completed by jaded and busy seniors every spring/summer is matched by their glacial paced apathy in replying. The specific number of ever changing hurdles that need to be jumped each year is arbitrary. How can this stiff system be equally correct to appraise an academic trainee and an acute medic?

    Adequacy
    The appraisal process is designed to demonstrate adequacy and highlight areas of failure. Doctors tend to be hugely competent over achievers who intensely dislike being told that they’re ‘satisfactory’ – always hoping that an appraisal may be a time where they may receive.. err.. praise. It doesn’t improve the quality of teaching I receive. It doesn’t remove the burden laid at my feet to ensure I have fulfilled the curriculum. It doesn’t prompt me with clever ways that other trainees have overcome similar problems or fulfilled ‘competencies’ (urgh.. shiver.. I hate this word).

    Not evidence based
    Reflective trainees may be safer than those who have no self-doubt, self-criticism but there is no evidence that asking someone to type their thoughts ensures that they become a safe physician. Forced reflection is public flagellation- it is phoney repentance for the cameras. I resent the reliance on faddy educational gimmicks such as reflective learning. Introspection may produce wiser decision makers – but the process of sharing isolated pockets of this cannot be proven to ‘improve’ the quality of doctors nor identify those who are failing.

    Evolution
    I am not afraid of change – I am ‘of the new system’ – a pilot F2 and a ST trainee. I resent the evalngelism for the current electronic portfolio because it is inferior to my paper based portfolio. My papers, thank-you cards and hand written notes are awkaward to integrate with the eportfolio system. Certificates, handouts and handwritten notes made my traditional portfolio ‘live’ and better reflect me. My paper portfolio grew over the years as I worked through medicine and into neurology – and now it is ‘lost’ and instead I have an empty emotionless electronic form to fill. Scanning my paper portfolio is an option – but an unappetising one. And let’s be honest – nobody but me was ever interested in my portfolio anyway. It was loved but unread, a living thriving accurate diary but as secret as a tomb.

    The failing doctor
    ARCP and the eportfolio will be used as the tools to ‘identify’ the failing doctors – but said tools are so blunt that they will not be identifying any more flaky physicians or dud Docs than the previous system of covert surveillance and nous could spot. If it is designed to identify those doctors who will be arrogant, reckless, or cut-corners – well the system cannot spot borderline psychopathic tendencies – I predict they will do equally well under the new system.

    The alternative?
    A trainee would have three questions asked of them and completed by three Consultants who feel they know the junior well enough to answer them anonymously; Are they currently up to scratch? Are they a good egg? Would you want them as a Consultant colleague? If the answer to any of those three is not a resounding ‘yes’ then they are put on probation (and by all means this probation may that start off looking like the current system). You must also report all complaints made against you. You must complete a truly anonymous 360 degree survey yearly.

    Babies and bathwater
    I do like the concept of 360 degree feedback (albeit I would make it so that you could not choose who completes this for you. The ability to cherry-pick those with a similar world view undermines the potential of the survey). I am aware that there is no room for negotiation here and that only by financing this industry can I become a consultant. I have to agree that they can share my data – which includes all my personal data – as an essential part of enrolment. At least I can opt out of the ever prying Facebook; and that the social media behemoth doesn’t require regular penitent reflection.

    • Thank you for your post. I share many of your feelings, in particular towards the “empty, emotionless electronic forms” in comparison to your “living, thriving, accurate diary.” I think we should all accept that the paper portfolio is dead and move on. The forests of this world deserve not to be massacred in the name of a mini-cex. However, I have a (naive?) hope that the elements of the paper portfolio that you cherished can be recreated in an electronic version. I cared for my paper portfolio, lovingly inserting section headings, summary pages, page dividers and highlighted passages from 360 degree appraisals. Of course not everyone did this, but I would welcome the ability to curate and showcase my achievements in electronic form. I’m sure others would agree.

  5. “I begrudge the cost of this static form; paying over seven hundred pounds for something which does not specifically reflect me or my training needs”. @Dr_Rhys do you know how much of your fees gets spent on ePortfolio? £18 per trainee, per year. Perhaps you should get in touch with your college and ask them where they are investing the rest of your fees.

    With more investment we can improve on technology, performance, apps, usability etc.

    Ben, ePortfolio team.

  6. It’s great the discussion has been opened up. In response to a few @Dr-Rhys comments (all of which are relevant btw) we do need to be very clear that e-portolio is an information portal for a system of work place based assessment. All of @Dr-Rhys comments are grounded in the fact it is difficult to translate the educational theory behind WPBA into a postgraduate medical arena. There are TWO debates here and its important they don’t get muddied. Blaming e-portfolio for the potential ills of WPBA is like blaming the taste of coke on the plastic bottle it comes in……

  7. Wai Keong Wong

    NHS Hackday is thinking of the eportfoilio as a project for the next Hackday. Have a look here for some discussions and ideas:-

    http://wiki.nhshackday.com/wiki/NHS%20ePortfolios

    It would be cool if the NHSeportfolio tool allowed me to: (subject for another post?)

    Suggestions by me: (@wai2k):

    – Has as inbox that allowed me to email it to create reminders assessment. For example:- I’ve just done a Lumbar Puncture and want a Direct Observed Procedure assessment (DOPS) done. The current process requires me to find a computer, login, click 3 buttons or so. If I don’t do this immediately I would forget. Instead, imagine an ability to just email a pre-defined email address like dops@nhseportfolios.org with the Subject: ‘Lumbar Puncture.’ This would create a placeholder DOPS for you to return to it later when you have the time.

    – Send me and my supervisors reminders if I’m clearly not performing enough work-based assessments.

    – Ability to email attachments that will automatically form part of my document and evidence library. Example use case: Thank you letter from patient. Photograph with smartphone -> Email to file@nhseportfolios.org with appropriate subject title. Done :)!

    – Ability to save the state of an assessment so that each one does not have to be completed in one sitting.

    – ‘Clever’ pdf forms that allow offline inputting of information that be uploaded to the NHS eportfolio when convenient. This is how ‘companies house’ does its annual submissions.

    – A mobile app. To make all the above easier. I guess this depends on how well the software of NHS e-portfolio has been written.

    • Thanks for these great suggestions. I have added them to “The tech improvement shopping list” (see menu at the top). Any more ideas very welcome!

  8. Comment from daniel_e_yarwood. Moved here as fits better than where originally posted.

    I am yet another trainee (ACCS) who finds the ePortfolio very frustrating. This is nothing to do with technical issues. This is due to it’s very existence which I believe unnecessary and burdensome. I have just sat through the introduction to my training programme and seen the amount of work involved in producing an ePortfolio and heard the consultant who is one of the training programme directors repeatedly call the whole thing a tick box exercise. So I sat there, blood boiling, learning just how much work I have to do in order to complete this tick-box exercise.

    Prior to medicine worked for a company called General Electric (GE). Like all American corporations they had a mission statement. Part of the mission statement was ‘Hate Bureaucracy’. They were a company of over 200,000 employees. They recognised in order to stay innovative, profitable, able to serve their customers etc., everything they did must be as stream-lined as possible. They were regularly recognised as the best run company in the world by all the surveys of that kind of thing. ePortfolio is, first and foremost, an exercise in bureaucracy. It serves no purpose other than to make the career of whoever suggested it and to provide a paperwork cover to imply competence (in fact it implies a competence at doing paperwork and playing a bureaucratic system).

    Why were GE the best managed company in the world? Because they spent thousands and thousands developing their managers. Early in the company’s history they recognised that if they were going to succeed, they needed to develop good management (what we would call one of the pillars of clinical governance). They had their own management development facility and research department. As a manager I was sent on month long courses, I was given opportunities to work all over the world. This was not exception, I was an average manager, this was routine. If I made a mistake? Money got wasted, no-one died or was irretrievably harmed. Now I know the NHS is not a multi-billion dollar company but I think the principle has to be the same if not more so. If a doctor effs it up, the stakes are far higher and yet our training is an amateur-ish, hit and miss affair that we and our seniors are far too busy to pay proper attention to.

    What is ePortfolio supposed to deliver? I have never been entirely sure. Here is what I think and what I have gleaned from my limited reading.

    1. Improve self-reflection

    Does it do this? Everyone knows that in the week before ARCP/RITA/sign-off trainees go and write ten pieces of reflection. I would argue that actually most reflection is done either personally after seeing a patient (immediately or lying awake fretting at night) or in a group of colleagues. Reflection is not a formalised process. This is anecdotal, what I have observed of my own behaviour and that of peers.

    2. Measuring competence

    ePortfolio provides a measure of one’s overall competence? I would suggest this is at best patchy and at worst wholly inaccurate. The clever trainee only asks to be assessed on cases that they have dealt with competently and by seniors that they have a good relationship with. The trainee who does not do this soon learns how to manipulate the system. The eportfolio then is about demonstrating an ability to play a bureaucratic game.

    The second question regarding this, if we have the eportfolio, why do we need professional exams? Or vice versa. Why do I need to sit PACES or MRCS part B if I’ve been certified competent at examining patients on ePortfolio? I suspect that it’s because most people believe that a tick-box exercise is no substitute. If it is no substitute, why are we forced to do it?

    3. Identifying trainees that are struggling.

    I actually agree that it does do this. It also gives one a (supposedly) objective way of measuring one’s competence in case it is called into question. However, there are two points here. Firstly, everyone knows the trainees that are struggling. The trainees who are struggling know themselves a lot of the time. Why does every single trainee have to fill in reams of unnecessary paperwork just to identify those who are struggling? If one’s abilities are called into question, why are we relying on a system that doesn’t prove ones competence as a doctor, rather it proves one’s competence at filling in paperwork.

    I would imagine that, in terms of tracking trainees it is ineffective. I don’t know how many times a consultant, unprompted by a trainee, casts their eyes over trainee’s ePortfolios, but I imagine it is twice to three times per attachment.

    4. Encourages self-directed learning

    I see no evidence of this. In fact, I would argue it does exactly the opposite. We are given a strict set of requirements we have to fulfill and an exact number of assessments we have to do on precisely specified topics. There is nothing self-directed about the ePortfolio.

    5. Produces a higher standard of trainee

    This again I strongly doubt and can find no evidence for. How does one objectively measure this? If many consultants are to be believed, medical training has never been so poor. As far as I am aware there is only evidence that the MSF provides any objective evidence but am not really sure.

    6. Encourages more teaching

    Again, I see no strong evidence for this. I would argue that the ePortfolio has replaced teaching. It is a way of giving consultants a way out of delivering any teaching.

    Is there anything it delivers? I am not sure of the history but I assume this has been adopted since the enquiries after Shipman as a tool to reassure the public that doctors are competent and are being closely monitored. I think it delivers this in a lazy, unimaginative and ineffective fashion, showing that doctors (intelligent and able mostly) are able to play a bureaucratic system.

    What then do I advocate? Well, I’m not sure. I would love to see clear evidence that ePortfolio works, that it is a good system. Other trainees (and consultants) roll their eyes, look into the middle distance and try and focus on their happy place or simply rant and rave whenever I try to have a constructive discussion with them. The best advice I had was my first consultant who said I should just accept the status quo because it wasn’t going to change. At the moment, it seems very much like the emperor’s new clothes. Collectively we all know the emperor is naked but no-one has the courage to stick their neck out and say so. If there are people campaigning for a better, less bureaucratic system, who are they and how can I join them? What can I do to help. I think the questions of apps and open source are missing the point. We are assuming this system is in some way a good one, a point of view I have heard no convincing arguments for. Being forced to pay for this system makes me want to cry quite frankly.

    Random thoughts that occur to me is that there is no substitute for consultant-trainee face-time. Why can’t we have an Oxbridge type tutorial system? A consultant has to spend one hour a week with each of their trainees. We all have a duty to do this under the GMC guidance and I am generally less than impressed by the amount that goes on. The current excuse for not teaching is self-directed learning or getting the trainees to deliver classes.

    I realise this is a double-edged sword. Being a good consultant and a good teacher are two different skills but then maybe this needs to be formalised more and those who are good teachers and committed to trainee development should be involved in this rather than every single consultant (who sometimes have less than no interest in their trainees).

    Through my foundation years it struck me that the only good, well-thought out piece of training that I had was ALS. Why? It was a great mix of pre-course work, theoretical and practical classes and thorough examination. All trainers had been on a training course themselves in order to deliver the training. In other words, it was well-delivered, well-run course. Why do we as doctors settle for anything less? Why is other training not delivered with the same thoroughness and gusto. Why can’t I become a certified immunology trainer? Why aren’t there similar courses in other areas?

    I would suggest a formal tests of competence throughout training that are either summative or formative in place of collegiate exams? Costly and time-consuming? Well, e-exams are perfectly possible and, if we are going to spend money, surely training and developing consultants/GPs of tomorrow is worth it? Why do consultants take on trainees if not to develop them?

    I hope the above does not sound too much like a rant. If it does I apologise but it does show my strength of feeling about the current state of training.

  9. Pingback: NHS ePortfolio Workplace Based Assessments in Core Medical Training – are they educationally useful? | Fitz' blog – Medicine et al.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s