Evaluating Crew Competency

Over the years, our customers have told us that the results obtained in a typical crew competency evaluation lack sufficient detail and evidence to help define and implement solutions for improvement.

To lead to implementable solutions, results must be traceable to, and based on, quantifiable facts and data. Therefore, all observations, evaluations, and conclusions should be supported by specific recorded evidence and a comprehensive contextual description. 

For example, what are the specific observations, facts or data that support the conclusion?

• Is it an anecdotal or a specific incident?
• Is it pervasive or isolated across teams, shifts, customers, third parties?
• Is it consistent, occasional, random?
• Did you witness it, read about it, or hear about it?
• Why were you observing it – what relevance does it have to the project goal?

Crew competency evaluations are by their nature subjective and based on observed behaviors during a limited period. However, by using a consistent behavioral evaluation framework, objective conclusions can be made about how the crew, and individuals on that crew match up to the customer’s expectations, as well as experience with other crews and industry norms.

An effective behavioral evaluation framework uses direct observation of processes, procedures, competencies, and behaviors. The desired result of the observation and evaluation is to establish — through evidence — the impact on:

1. The overall quality of the work performed
2. Efficiency and effectiveness of the work
3. The number and duration of non-productive time (NPT) incidents

In executing the behavioral evaluation framework, an Athens Group Services Surveyor would observe, audit, document, and evaluate six behaviors on the operating rig:

1. Conformity to and any deviation from policies and procedures, fleet norms, applicable standards and regulations, and general industry and domain best practices, considering:

• Are there deviations from procedure or best practice? 
• What exactly are those deviations and how are those deviations tied to NPT?
• Is there a “normalization of deviance” occurring? This happens when people within an organization become so accustomed to executing nonconforming practices without consequence that it no longer feels like a risk.
• Are there differences as to how the rig interprets and implements any uncertainty or flexibility inherent to the procedure or best practice?

2. The skill and competency levels, work habits, interactions, attitudes, and behaviors of the offshore crew, considering:

• Technical/job competency – does the crew member have the specific skills and experience necessary to successfully complete the task?
• Morale and work habits – does the crew member have the necessary attitude to successfully complete the task? For example, you can be the most technically competent subsea tech on the planet, but if you are demotivated, have no sense of pride or ownership, or are just disinterested in the job, this will reflect in the quality of the work.
• Efficiency, which is a measure of how much effort is put into getting the job done – less effort for the same work accomplished is more efficient.
• Effectiveness, which is a measure of how successful the work is, i.e., did the work result in a quality output? For example, one can be very efficient – rebuilding a valve in less time than anyone else – but not very effective if those valves have a very high incidence of not working or premature failure.

3. The maintenance, repair, and defect/failure analysis habits of the crew, considering:

• Are they just part swappers or are they curious about why something failed or what the defect is?
• Do they do a defect/failure analysis?
• Do they note the condition of regular/periodic maintenance parts – condition, % wear, if the part needed to be replaced to feed data to the maintenance system for plan/schedule/period improvements?
• What are their actions when they pull a part, and then replace it?
• Do they pull from stock and confirm the replacement kit looks like the part they just removed?
• Do they carefully disassemble the old part to maintain the ability to analyze condition or defect?
• Do they take the time – are they interested in what the defect was and the condition of the part?

4. The influence of the activities, interactions, and behaviors of the rig leadership on the activities, interactions, and behaviors of the crew, considering:

• The management/leadership environment. How are the rig teams managed, and how do the teams respond to that management? Are the leadership hands on, overbearing, passive or disconnected? 
• The support environment. How are the rig teams supported by the onshore technical team, and how does the offshore team respond to that support? Is the support team hands on, overbearing, passive and disconnected? 
• The “management by wandering around” factor. Are each of the leads managing from the appropriate locations and how are they using their time? Do they go out on the floor to see what’s going on? Are they involved in the day to day, or do they manage by fiat and message from the office? 
• Decision making. There may be key differences in who makes decisions, when, and how empowered the crew feels to make decisions based on the relationships with rig and subsea team leadership.

5. The influence of the activities, interactions, and behaviors of the third-party surveyors and customer representative on the activities, interactions, and behaviors of the crew, considering:

• The customer/third party environment. How does the customer/third party interact with the management/leadership and the crew? Is it hands off, direct hands on, or micromanaged? How does the management/leadership and crew respond to this?
• Decision making. There may be key differences in who makes decisions, when, and how empowered the crew feels to make decisions based on the relationships with the customer representative and third party.

6. The design and use of on-board systems for scheduling, maintenance, repair, spares management (availability, storage, labeling), and configuration management.

• Do they protect and store an old part in a way that allows defect/failure analysis?

Athens Group Services has developed a comprehensive and efficient method of executing crew competencies based on observed behavioral evaluation. Contact us at info@athensgroup.com to discuss how you can evaluate your crew competencies and improve the efficiency and effectiveness of your rig operations.

To see the latest Athens Group Services developments, please follow us on Linkedin and Twitter.