Applied Process Engineering Best Practices

Guidance to analyse data and make improvements

This website, and maybe one day a complete book, is going to take you through some best practices when applying process engineering to identify issues, find solutions and then sustain those improvements. The focus will be on leveraging data from your facilities in order to do this. It's a broad topic covering many areas, and likely to take me a long time to get round to all of them.

Why am I writing this?

In my experience I've found there aren't many process engineers who understand these practices and there's no-one around to show them. Everyone seems to be reinventing the wheel every time. I've not worked everywhere, or for every company, and I know there are a lot of good engineers out there who are already doing all these things. Watching conferences related to this topic suggests there's still opportunities to share these best practices with a wider audience.

I also want to write this for myself. Everyone says teaching others is the best way to learn and writing things done makes me really think about them.

Software selection

Throughout this series of posts I'm going to make reference to software packages. In my career I've used certain pieces of software, that's not to say that these are the only ones available, but these are the ones I'm most familiar with and so I'll refer to them a lot. They aren't uncommon and certainly things like the PI historian are widely used, but I appreciate there are others. I'll use specific examples from these pieces of software to demonstrate my point. I'm afraid it'll be up to you to figure out how to implement it in the software you have available. Alternatively, it may give you business case to justify piloting/licensing new software.

Key software and their scope

To set it out from the beginning this is what I've used and will refer to:

OSISoft PI Stack

This is the process historian, storing all the instrumentation data (pressure, flow, temperature, etc.) and other time series data from the facilities. This is the foundation for everything, as the process engineer, this is where you will start from.

The PI stack is more than just the historian, it also includes PI ProcessBook, PI Asset Framework, PI Vision, PI data logger, and PI integrator for business analytics. Not all of these come as standard, and you might not need all of them but they're important parts that are often less well utilised.


Tibco Spotfire is a general purpose visualisation tool, it can bring in data from many sources, undertake transformations and calculations, and create visuals so effortlessly. This is one of many visualisation tools out there. Whilst similar, they're not all the same and often have very different capabilities. One common competitor that gets pushed is Microsoft's PowerBI. Certainly it seems loved by corporate IT management, especially if you're an Office 365 user, and so far (in 2020) it's capabilities, in my opinion, aren't very well suited to time series analysis. It can have its place but it appears to be better suited to more typical transactional data or summaries and reporting.


Lab data SOR

Operator logs

Text input from shift logs. Non-numerical data.

This one is somewhat vague right now. It could be a packaged solution such as Seeq, or Trendminer, or it could be more homemade solution built on a foundation of a programming language and available libraries such as python or R. The first two are perhaps more suitable given they're managed by someone else, but you may not have the luxury of being able to afford multiple pieces of software (although I'd argue with the right application of them you will more than easily pay for them). Learning a little python or R can go a long way, although unless you put in a lot of effort together with your IT department, they'll likely remain personal productivity tools.

There are likely others, some may take the form of a paid for service by a “data science” company. Typically in this instance you feed them data, they run their tools and you see the output. More than likely they are using python and available libraries plus their own implementations or customisations to do this. You'll be paying for software and people's time, so it could be more expensive. However, you do get their expertise but you may never learn anything yourself. They are also a business who's objectives are to make money, so be aware of conflicting priorities.


Like every toolbox, there are lots of tools, you perhaps have a few favourites but then some others you might bring out from time to time for a specific problem. Some of those might include:

  • Excel – everyone has it, everyone knows how to use it, but you can't build a systematic
  • Control systems tuning/optimisation – many vendors provide tools to identify poor control loop. These are very specialised but do a great job, and can help with tuning loops. You might not be fortunate to have this so I'll cover some simple monitoring to help spot issues.
  • Turbine/rotating equipment software – like control loops, vendors of turbines and other specialist equipment may have their own software for analysing the performance of the equipment. Probably used more by reliability/rotating equipment engineers.