How to harness machine learning for smarter dry bulk data processing

Understanding machine learning, where it can be used, and how to get started

What is machine learning?

You might not have heard of machine learning, but you’ll probably have heard of artificial intelligence – you may even have seen the films telling apocalyptic tales of the rise of the machines. But in the real world, what exactly are these machines and what do they mean for us?

The term artificial intelligence (AI) was coined back in 1956 by computer pioneer John McCarthy, who described it as the science and engineering of making intelligent machines. It is the capability of machines to imitate intelligent human behaviour and decide things for themselves – acting without being specifically programmed to do so. Despite the many claims and regular news stories, true artificial intelligence still has a very long way to go before becoming a reality.

If computers aren’t being programmed to perform specific tasks by intelligent humans, how do they build their artificial intelligence to act alone? Of course, they must learn for themselves and create rules from the raw data that has been fed into them and that’s what we call machine learning.

The concept of machine learning (ML) was developed by computer scientist Arthur Lee Samuel in 1959 and has advanced so much in the intervening years that it is now being used in many popular software applications.

Just to confuse the matter though, technology companies today use the terms artificial intelligence (AI) and machine learning (ML) interchangeably – often saying software is powered by AI when it is in fact using ML. Put simply, machine learning is a way of achieving artificial intelligence.

How to work smarter not harder

Machine learning allows software applications to get smarter the more they are used and the more data they have to hand. A good example of this is predictive search where Google will autocomplete your search term and make suggestions based on what you are typing and what other people have searched for. The real-time product recommendations made by eCommerce platforms like Amazon are another example of machine learning that you’re probably already familiar with.

In a commercial setting, having computers focus on data processing can really help improve operational efficiency. In shipbroking specifically, there is one major way that machine learning can be useful. Software can now automatically process open tonnage, cargo orders and fixture information from a range of communications – including email messages, email attachments and instant messages – which frees up time for specialists to do the things that computers simply can’t. Giving senior professionals more time to do the hard part of the job and add real value – to analyse the consolidated data and combine it with over-the-phone information to provide invaluable advice to clients.

In our recent article Future focus: How new technology is changing the role of shipbrokers, we explore other ways that modern, intelligent software is being used in the dry bulk shipping industry to help improve both efficiency and team collaboration.

Who is already harnessing this technology?

As we mentioned above, the term artificial intelligence is used quite loosely by many software providers and so is the term machine learning. Performing standard tasks faster than any human does not imply that a system is intelligent. A computer having processing rules that need to be programmed and amended by humans is not the same as machine learning.

Historically, rule-based systems have been used to extract information from unstructured data sources – such as email and instant messages – but these systems had to be taught. If something came along that didn’t fit the pre-determined rules, the system would break and would need to be fixed – by a human. Fixed rules like these can also generate mistakes. When a system indicates that something is the case when it is not we call it a false positive. For example, the system may recognise a ship’s flag as a country name – which is smart – but then go on to map it to the open position of a ship – which is not. So, the errors need to be continually corrected and the rules need to be regularly maintained – and this requires an army of people.

At Shipamax we use a supervised machine learning (ML) approach instead. We have an ever-growing dataset that we use to ‘train’ the system and have a custom-built algorithm that uses advanced mathematics to detect whether information is correct or not. Whenever something incorrect is identified, our supervisor steps in to provide the correct interpretation and the machine learns from this intervention – it will avoid making the same mistake next time. It has got a little smarter.

The power of data in dry bulk shipping

The fast-paced nature of the shipping industry means that no-one has the time to input every piece of data into their systems – even if analysing that data would potentially provide added intelligence and real value. While this trade-off made perfect sense in the past, as the shipping market continues to wake up to the possibilities of technology, this decision may not be the best strategy for the future.

Using data science, intelligent systems and machine learning to process and analyse key information and combining this with a broker’s personal knowledge of the market can unlock powerful new insights and value to their clients. Similarly, charterers exploiting data science and technology in the same way have the ability to advance their trading strategies in imaginative new ways.

There is a window of opportunity today for progressive companies to take advantage of developments in machine learning technology and turn their data into a competitive advantage.

How to get started with machine learning

To start harnessing the power of machine learning, you first need to determine which parts of the business would benefit most from this technology. Our article How to bring your dry bulk shipping process into the digital age outlines some simple steps that will help you uncover areas with the most potential for change.

Next, you’ll need to decide whether it would be best to build the machine learning capability in-house or more advantageous to work with an external company or specialist provider.

The ‘build versus buy’ decision is never an easy one. If the opportunity for machine learning is limited to processing data – for example data held in high-volume communication systems such as email – we would invariably recommend working with a specialist partner. Alternatively, if the opportunity lies in manipulating pre-processed data to create value-added advice and to develop new trading strategies that are unique to your organisation, it may be best to keep things in-house.

If you do decide to work with a specialist, you should ensure they have genuine machine learning (ML) credentials and are not – as we discussed above – just putting a trendy new name on an old rule-based system.

At Shipamax, we work with some of the largest shipping companies to process and clean their unstructured data so it can be reviewed and analysed by experienced team members internally to add more value. Using our intelligent layer and machine learning technology, we connect a multitude of unstructured data sources with a range of structured systems. Our unique web-interface also allows brokers and chartering managers to migrate away from old ineffective tools – such as MS Excel – and track data that was previously un-trackable.

We'd love to talk

Shipamax helps free logistics companies from manual admin using plug and play automation. You can find out more about how we're helping shipbrokers, by using this link to book a demo with a member of our team.