Tag Archives: app

The RCP are listening – what do you want to tell them?

A few months ago I participated in the first Royal College of Physicians (London) ePortfolio reference group. This group has been established to increase the trainee voice in the ePortfolio development process, and should be celebrated and welcomed. The RCP is leading the way in engaging trainees to make sure their educational experience is as good as possible. Does your College have trainee representation? If not, you should ask them why not…..

Screen Shot 2013-05-20 at 12.29.28

At this meeting we discussed many areas, and tried to keep things positive, focusing on what changes could be made that would make the ePortfolio better for trainees. The RCP agreed to take these issues forward to a meeting with the NHS ePortfolio providers NES:

1. Development of an API to allow integration with third-party tools

It is great that the RCP were willing to take this on and discuss this as a priority with NES. There are lots of apps being developed (including many at NHS Hackdays) which help trainees to capture their learning, for example in log books. There is currently no way that this data can be incorporated into the ePortfolio. Since NES does not have the capacity to create additional software, allowing third party software to interface would be a major step forward in increasing the functionality of the ePortfolio.

2.  HTML5 app to concentrate on work place based assessments

As you may be aware there is now finally an app! Amazing! But at present it can only log reflective practice and send tickets. The group agreed that WPBAs/SLEs should be the priority for the next release of the app.

3. Development of summary views of data to support ARCPs

The current summary overview is unimaginative and unhelpful for ARCPs and PYAs. I have heard too many stories of trainees still having to print their portfolio, or send documents separately. In my own ARCP last year the panel could not find all the evidence they needed and I had to send it as PDFs afterwards. There should be a better way for the trainee to curate the content of the ePortfolio on an ARCP page. It should be possible to define the requirements for ARCP fulfilment and then demonstrate how these have been met. The data should be displayed in an easy to view at-a-glance way so that ARCP panels can rapidly see and access what they need.

4. Change to how ePortfolio roles are created so they are cumulative

Rather than having to switch between roles and select one, with different access levels for each type of access account. This is particularly important for those with significant assessor roles in addition to trainee roles.

Please let us know your thoughts on these issues, and tell us what your priorities are so that we are well prepared for the next meeting, hopefully in June/July. I’m looking forward to hearing how these issues were received and what plans are in place for taking them forward.

We have a real opportunity to raise issues, focus the priorities of those with the purse strings and effect change. Don’t stay silent! 

You can give your feedback in a number of ways:

  1. Comment here (below this post)
  2. Talk to the JRCPTB ePortfolio team
  3. Talk to the your Foundation Programme school
  4. Give feedback to NES directly on Twitter
  5. Give feedback to NES about the app specifically 

There’s an app for that

After much complaining it is great to be able to be positive about the NHS ePortfolio. Developer Ben from NES has been working on an app which is now available in beta version. It is html5 so whether you’re an apple or android user it should work on your device. You can download it right now, have a go and provide feedback to make sure the next iteration is even better.

Go to this site on your device: app.nhseportfolios.org  and login with your usual password.

At the moment the functionality is limited. It supports reflective logs and ticketing, but does not allow offline completion of assessments such as mini-Cex. I also could not get it to store data within the app to synch later initially as it made me synch before I could logout, or data was lost. I also got stuck inside the app and couldn’t get back to the rest of my iPhone. This was a bit of a nightmare – stuck in the ePortfolio forever! It would be great to hear if anyone else had these problems or if it was just me! I have since managed to use it without crashing – phew! 

You can send your feedback to:  app@nhseportfolios.org 

An app is long overdue and does not fulfil all of our ePortfolio dreams yet, but it’s a start and should be celebrated. Hopefully, by testing and getting feedback before further work, the app will fulfil the needs of users and make logging evidence of our skills and learning experiences fit the realities of our working lives. And perhaps this agile way of working might permeate back to the rest of the ePortfolio. A girl can dream…..

NHS Hackday and the oPortfol.io

This weekend I was at NHS Hackday. Doctors, other healthcare workers, students, patients, organisations and software developers came together in their free time to make stuff that could make the NHS better.

Thanks to @londonlime

Thanks to @londonlime

I was astounded by the last Hackday. I didn’t think my expectations could be surpassed. They were.

The projects were diverse, aiming to solve everyday problems at all levels of the NHS. You can see the list on the Google doc and they’ll be on the wiki soon. The highly deserved winner was OpenHeart. The team used the amazing open source electronic health record at Moorfields Hospital, Open Eyes, and adapted it for use in Cardiology. The end result was stunning. It will save hours of doctors’ time, will create patient records that are much more understandable for patients themselves and for GPs, and will improve communication and therefore the quality of care.

Another favourite was Dementia scrapbook, an app to allow family and friends to contribute to a virtual scrapbook of memories and reminders. It has an easy to use touch interface that can be used by carers or people with dementia themselves. Dementia is common and this takes a very patient-centred approach to solving problems many of us may face in the future. I hope to see it available soon on the app store.

Cellcountr, initially hacked at the Liverpool hackday, was built on with additional features such as data visualisation and a customisable keyboard. It will be launched in the next month at a Pathology conference, and will then make a real difference to doctors, and their ability to accurately diagnosis patients with haematological conditions.

So what did we do? We created oPortfolio, an open API which allows trainee doctors to record learning events online, offline and on the go. It includes a webapp, a mobile-friendly site, an iPhone app, and an android app that all synch data. From nothing to all this in 36 hours! The team were incredible: full of talent, patience, and creativity.

Oportfolio

What does it do?

It solves an immediate need to log learning events on the go (see examples below). It lays the foundation for a more complex system to log assessments and meetings. With (quite a bit) more work it could be a simple open portfolio that doctors who are not currently in a training programme (eg LATs, people doing fellow jobs in between F2 and speciality training) could use to track their professional development. The funding model would have to be clarified as development and hosting is not free! It could also be an arena to experiment with and showcase new ways of organising a professional portfolio that could usefully feed into the debate on what and who a portfolio is for. It could highlight how different systems talking to each other and 3rd party apps and plug-ins have the potential to improve a core product. Another fabulous creation was Quicklog, an app to log personal development in performing procedures on the go. They built in data visualisation to encourage reflection and chart progress. It would be fantastic if the data from Quicklog could be integrated into a portfolio system. Anyone who is interested (and understands it!)  should look at the code on github for oPortfolio and Quicklog!

Screen Shot 2013-01-28 at 11.55.34

What does it not do?

It is not a replacement for the current ePortfolio system(s). The NES ePortfolio and others (eg surgical portfolio) are complex structures build up over years, with thousands and thousands of pounds of investment. Many have questioned whether they are value for money and I can’t answer that but good software does cost. Existing systems have  layers of access rights and methods of data extraction since these were priorities for the bodies who paid for them. They have cloud hosting and data security. People have spent years making them do what they do and it would be crazy to think they could be replicated in a weekend. They have their problems and must be improved but they are here to stay until a better alternative exists.

The focus of building a model Oportfolio was the user experience. If it was developed further it could fulfil a need for trainees who are not in a current training programme, who currently use various cobbled together documents on Evernote, phone notes apps, word documents and paper to record their learning and showcase their achievements when applying for jobs. With regular end-user input it could be beautiful, and a joy to use!

I am sure that our exploits this weekend will appear highly challenging and controversial to some. But I am not controversial. I have always highlighted the frustrations felt by trainees (which are well known) but advocated for engagement with all interested parties: individual trainees, trainers/educational supervisors, LETBs, Trusts, Royal Colleges, current ePortfolio provider NES, the GMC and HEE. We need to get our heads together and think about what the future of training will look like, what tools are needed to enhance learning, and how they will be funded.

The NHS can’t keep putting up with unintuitive, inflexible IT that doesn’t match the realities of practice. As demonstrated at NHS Hackday; intelligence, enthusiasm, creativity, a few humous sandwiches and some coffee can create magic. But that magic needs support and investment to make it sustainable. Muir Gray says change in the NHS will come from the bottom up. He is one of a few inspirational people at the top supporting projects in which frontline staff make a difference. We could do with a few more like him….

News from the trenches

If you have not yet already seen the blog by Dr Fitz “NHS ePortfolio WPBAs in CMT: are they educationally useful?” I recommend you make yourself a cup of tea and have a read. He has taken a significant amount of time out of his day as a medical trainee to document his experiences of using WorkPlace Based Assessments (WPBAs) in the “real world.” He has done this not because he likes to rant, but because he is genuinely interested in how he can support his own learning, and ensure that the assessments he undertakes are valid and useful. He is like many of us – he wants to be a better doctor – but wonders whether the system currently helps or hinders this.

teacup

Dr Fitz seems representative of many Core Medical Trainees (CMTs). In 16 months he has undertaken 51 assessments (ACATs*, CbDs*, miniCEXs*, DOPS* and Teaching Assessments) and 2 rounds of MSFs* (360 degree assessments) with 35 responses from colleagues in all. This is a little more than the minimum requirement for his ARCP (annual appraisal) and is a good bank of data on which to reflect. Well, it should be. Dr Fitz looks in detail at the contents of his ePortfolio and wonders what it really tells him. Of particular concern is the documented feedback that is at the heart of these assessments:

“Unfortunately the stats don’t look good. Over the course of 9 ACATs, covering the management of 55 patients over 12 months, I received 127 words of feedback. That is 14.1 words per ACAT and 2.3 words per patient seen. About 6 tweets.”

He is quick to point out that this does not necessarily reflect the amount or quality of verbal feedback he received, but

“…the educational benefit of my on-calls and the instruction I received from my consultants was separate to, not part of, my ACATs.”

Sadly the story is very similar for his CbDs, miniCexs and DOPS. The quality of the feedback documented on his ePortfolio is poor, and is of no use to him when he reviews his ePortfolio to prepare for his ARCP and consider what should be on his Personal Development Plan (PDP).

His conclusion is balanced and reflects the feelings of many trainees who have commented here previously:

“Overall, my experiences of the NHS ePortfolio assessments for CMT is that whilst they may act as a record of learning, they fail to be a useful educational tool in themselves. This is mainly due to the discord between how they are supposed to be completed and how they are completed in practice. Teaching, supervision and education is happening, but it is in spite of WPBAs rather than because of them.”

enthusiasm for portfoliosHe doesn’t end there and has several suggestions for improvements. They mirror comments already on this blog and range from urgent technical improvements (really, do I have to mention an app yet again?), to faculty development. This is just the kind of input Training Programme Directors, and National Programme leads want and need. Trainees have reasonable and real concerns about their training, and are engaged and enthusiastic about improving it.

Harnessing this enthusiasm will be vital.

If you haven’t already contributed to the Shape of Training review please do so. This is a collaboration between several higher bodies including the General Medical Council, Medical Education England, the Academy of Medical Royal Colleges, the Medical Schools Council, NHS Scotland, NHS Wales and the Conference of Postgraduate Deans of the UK. The review is considering what changes are needed to postgraduate medical training to make sure it continues to meet the needs of patients and health services in the future. This includes options to support greater training and workforce flexibility, and how to address the tensions between obtaining training and providing a service. You have until February to make your voice heard.

* ACAT= acute care assessment tool, CbD = case-based discussion, miniCex = mini Clinical encounter, DOPS = directly observed procedure

What would you ask?

As a result of this blog, and “a little help from my friends” I have a meeting next week at the Royal College of Physicians to discuss the NHS ePortfolio. I hope to articulate the discussions on this blog, and those that I have had over email, twitter and in person with many of you.

I hope to communicate the frustration felt by trainees and others about the NHS ePortfolio, along with the hope that it could be something so much better: a true educational and professional development tool to support lifelong learning. I hope to gain a better understanding of the views of the Colleges, in particular the view of the governance structures, the funding streams, and the future vision.

Broad areas I hope to discuss include:

  • lack of user input on any decisions about the ePortfolio and proposals to change this
  • poor communication of the rationale for, and timing of changes (personally, I had no warning about the change to formative and summative DOPS) and the effect this has on demoralised trainees
  • technical limitations – such as speed, which increase frustration and decrease utility
  • design limitations – such as lack of flexibility, poor organisation and visualisation of content so that it is hidden away and can’t effectively inform PDPs. These design limitations have an adverse effect on the educational utility of the ePortfolio
  • educational limitations due to lack of an app, meaning delays between a workplace-based assessment being observed, and completed online (therefore feedback becomes meaningless)
  • other limitations – lack of ability to share content with anyone other than supervisors, lack of interoperability and exportability of data for more creative uses
  • what does the College believe is the answer to the question “who owns the data in the ePortfolio?”
  • is there scope for greater collaboration between JRCPTB, RCPCH, The Foundation Programme and other groups who commission their ePortfolios through NES, to invest jointly for change – an app would be a perfect opportunity for this. The users of these systems share many basic needs, and could have individual specifics built in
  • what is the future of the ePortfolio, in technical, and educational terms, and how will this be decided?
  • how is the ePortfolio funded and commissioned? What are the plans for the future? How can we improve the process and increase transparency?
  • the concept of an ePortfolio has great potential but is not delivering what is needed. Investment is essential. User input is essential. There is a great opportunity to get this right and improve both the perception and reality of postgraduate training. Who will lead this change? Will it be the RCP?

I am very grateful to the RCP for taking the time to meet with me, and hope it will be a very constructive and positive conversation. Please let me know what else you feel is essential for me to communicate, as we begin to get closer to a process for change.

What would you ask? Please let me know by commenting below. You can also still edit the ePortfolio Trainee Roadmap

An ePortfolio wish list – created by an NHS ePortfolio NES developer

There have been lots of great suggestions on this site as to features that would make the NHS ePortfolio better. There has also been some insightful debate on the model that we should propose for the future, from aesthetics to functionality to commissioning. When I find a free hour I will make a summary page, as the debate has extended far beyond my initial expectations, and on arriving to the site it may now be difficult to see a coherent narrative. Some ideas for improvements are summarised in the tech improvement shopping list, others are amongst the comments on various posts.

A new development is that an NHS ePortfolio NES developer @ben555 has started a wish list where you can suggest improvements and vote them up or down the list.

Make a (NHS ePortfolio-related) wish

Whilst debate over longer term issues  continues, this is a great way to gather momentum for change. I have voted for an app – this is urgent!

Make your suggestion and cast your vote now…

NHS Hackday and the ePortfolio Data Liberation Front

I spent this weekend in Liverpool at NHS Hackday. I had no idea what to expect. I had never met anyone there before and only knew a few names from twitter and google groups conversations in the weeks running up to the Hackday. I wasn’t completely sure I knew what a Hackday was.

I was astounded. 

I spend a lot of my life getting frustrated by the slow pace of change and the massive inefficiencies in the way that we work. I want to be freed up to spend time teaching, learning, writing, thinking, talking to patients and providing care. I hate unnecessary paperwork and bureaucracy. I hate meetings that don’t achieve anything.

NHS Hackday was a breath of fresh air. A diverse group of people with totally different backgrounds, most of whom had never met,  got together, discussed problems and solved them. In a weekend!

I will describe here what happened at the Hackday, what our project “The NHS ePortfolio Data Liberation Front” achieved and why it won 2nd place. There is far more info about how it is run, by who, and why on the NHS Hackday site. You can also see an interview with Carl (from OpenHealthcare) on Youtube:

 What follows are my personal impressions.

The HACKDAY CoNCEPT

On Saturday morning, whilst people were registering and getting coffee all those with ideas for projects wrote them on a board. Everyone gathered in the main hall and each idea had 2minutes to pitch. After all the pitches, people gathered around signs indicating each idea, and people formed groups. Then the work began. Groups discussed their vision, their proposed solution, and thrashed out conceptual and technical details. Fuelled by enthusiasm, tea, coffee and wotsits, software developers created things out of thin air (OK, out of data and code, blood, sweat and tears). Health professionals like me, who couldn’t code, were on hand to give context to the projects and point out real-world hurdles, which could then be worked around.

The NHS ePortfolio Data Liberation Front

Our group consisted of me (full of ideas, no understanding of code), Nicolas Tollervy, a developer (a genius with lots of patience and an incredible ability to work round every problem the project presented him with)  and Marcus Baw, (a GP who can code a bit and is a RCGP Health Informatics Group member, who was a great bridge and font of knowledge on NHS informatics issues).

We discussed some of the problems with the current NHS ePortfolio and possible workarounds. Since the code is not open and there is no API this was no simple problem.

We discussed the urgent need for an app to make trainees and trainers lives easier, and make WPBAs educationally valid. Any app would have to be able to get data into the ePortfolio so that a WPBA showed up not just in the personal library section as any random document, but in the WPBA section. With no code and no API this would be a great challenge.

We decided to focus on the fact that my data is locked in a vault in my ePortfolio. Whilst it is in there I can do nothing with it.

I want to liberate it, as I could then do anything I want with it! Ideas include:

  • visualise my achievements and progression
  • present the data in a way that my supervisor can see, understand and give feedback on
  • present the data in a way that makes it clear I have achieved all the competencies required by the JRCPTB for ARCPs and CCT
  • integrate the data into my CV, my online CV, an alternative ePortfolio (mahara, Googlios etc), use it for job applications
  • allow me to take the data with me into another role (progression or change of career path) eg Foundation Trainee –> Emergency medicine ACCS trainee –> GP trainee –> GP (all use different ePortfolio systems)

Not only is there a practical need for this, but the more we talked about it the more I realised that this is bigger than practicalities. It’s a philosophical argument. It’s my data. About me. I want it liberated. I can already download a PDF so clearly no-one disputes the fact that the data is mine and I have a right to it, but a PDF is useless.

@ntoll worked incredibly hard (with breaks for coffee, sandwiches, a trip to the pub and a curry house), came up against many problems and found ways around them all. We modified our plan as we went along, and decided that the best use of our time would be to do a ‘proof of concept’ and focus on a particular data set within the ePortfolio (there’s a lot of data in there, and it’s not organised as logically as you might imagine!). By the time we reached the submission deadline of 12.00 on Sunday we had something to show for our efforts. @ntoll made some finishing touches and we put together a brief presentation.

All 15 projects that had been selected from the pitches presented (a strict 5min and 1min for questions) to a panel of judges including: @MarkPriceDavies (chair), Ian Gilmore, Dr Farath Arshad, Zeinab Abdi, Francis Irving @frabcus, Dan Lynch @MethodDan, and Lilian Wiles. They deliberated and at 17.00 announced the winners.

The Other Projects

You can see more details of the projects on the NHS Hackday site, and get all the code through the wiki and on github, since all projects are open and shared. There were lots of fantastic projects but those that particularly caught my attention were:

  • AskIt (a general purpose question asking android app for any questionnaire you need – Waterlow, MUST score, falls assessment etc. Simple, effective, important!)
  • Making sense of patient comments (data visualisation from sources such as NHS Choices – massive potential applications)
  • CoIncidence Gate: a Conflict of Interest tool (scraped data from conflict of interest statements on Pubmed – something like 480,000 papers analysed!! Again, follow the link for more discussion on the massive potential applications of this project)
  • BleepBleep (making in-hospital communication better. An end to having to call switchboard. An end to the bleep! I trialled this, and am keen to help get it into hospitals now! Stop wasting time on hold)
  • GAAG: Guidelines at a Glance (there are well-studied barriers to doctors using guidelines, meaning patients don’t get best care. GAAG provides quick access to personalised most-used bits of guidelines on an app. Lots of potential for social add-ons, highlighting when guidelines change, seeing what peers use, rating bits of guidelines. See presentation for more info. Can’t wait to use it!)
  • Bloodcount (haematologists sit at very advanced microscopes counting different normal/abnormal cell types using very un-advanced technology = clicker and pen and paper. Bloodcount is a desktop system of a counter with keyboard shortcuts, reference normal and abnormal cells, report generation and learning function. Hard to describe to do it justice. A worthy winner!)
  • wtfdoc (an NHS jargon buster for patients and relatives as an app. Has a database, and if a term is unknown it will crowdsource answer through twitter and other sources. V clever!)

Why I think We Won a Prize

Our project won the First Scraperwiki prize for scraping, and came joint second overall on the day. I think the reasons we won are multiple:

  • @ntoll achieved amazing things writing novel code to scrape data out of a closed system and generate a .json file of hierarchical data that could then be used. In just a day and a half this was some achievement!
  • our pitch was powerful as this is an issue for all doctors of all specialities at all levels, especially with revalidation now a reality. Facilitating learning for healthcare professionals is in all our interests as a society
  • the concept of data liberation goes beyond this project. Who owns the data in public databases? Who owns the data in the NHS? What right does an individual have to their own data? What right does an institution have to keep it from them?

What next

I owe a huge thank you to the organisers, supporters, volunteers and participants at NHS Hackday Liverpool 2012.  And a special thanks to Scraperwiki for providing prizes including my beautiful new Google Nexus 7! This weekend I saw innovation in action, providing real, practical solutions to the day-to-day problems facing those who work in and use the NHS. Some of these solutions are now in use – today! Others will be worked on outside the Hackdays or at the next one. I have had my mind opened to new ways of working and have returned to work today full of enthusiasm and inspiration.

There’s no going back now. I’m a doctor who loves geeks who love the NHS, and I have the T-shirt to prove it.