How Financial Risk Management Software Can Benefit from Big Data Analytics

How Financial Risk Management Software Can Benefit from Big Data Analytics

Written by Mykhaylo on August 2nd, 2017

The benefits of big data analysis for financial risk management explained with examples, use cases and commentary from experienced developers.

More data has been created in 2015-2016 than in the whole history of the human race. In fact, by 2020, there will be 1.7 megabytes of new data created for every human, every second.

Data translates to money. In fact, according to Baseline, a 10% increase in access to big data can result in as much as $65 million in increased net revenue.

Today, over 70% of companies are already investing in big data (or at least have a plan to invest in it). And here’s why. Of course, they see the increased revenue potentials. At the same time, they also understand that part of the increased revenue comes from using big data analytics for risk management. The more fraud they can prevent, the better; the more predictive analyses they can gather, the more they can determine creditworthiness; the more data is analyzed, the more potential breaches can be mitigated.

Two Simple Examples of Risk Management in Action (or Not)

Big data can and should be used to manage risk in investment management. Two recent cases show its importance:

  1. In 2012, Warren Buffett increased its investments in Tesco, a UK grocer, based upon company profit projections. Had he used big data analytics that was available at the time, he might have re-considered. Both structured and unstructured data (social media sentiment) were “red-flagging” the company. The end result was that Buffett lost $444 million throughout 2014, as he dumped the under-performing stock.
  2. More and more U.S. companies, large and small, are requiring drug testing before they hire new employees. This activity has become so normal, that investors took notice and began to gather big data on companies that were manufacturing these tests as well as companies that were incorporating their use. This gave them information on those drug companies that were “cornering the market” on sales and which ones were good investments. But going even further than that, the big data could be used to identify employment rates of companies, based upon their purchases of these screening tests. Declines in such purchases might signal other problems within a company, and further research before an investment is made or a decision to sell.

Making investment decisions and changing positions on current investments should occur in a much more data driven environment. Fortunately, big data analytics can give investors a framework to do this.

Applications of Big Data Analytics in Risk Management

Big Data Analytics in Risk Management software (1)

AIG, looking back, probably wished that it utilized big data analytics for risk management in the early 2000’s. It would not have miss-judged the risk potential so significantly and could have avoided at least some of its financial meltdown. And big data has come a long way since 2008.

Newer technology and trends now make big data analysis much easier to apply:

  • Frameworks and platforms for collecting larger and larger amounts of data, including longer histories will continue to be deployed and refined. This will allow far more accurate risk management analysis and predictions, particularly in the areas of investment management and the “health” of derivatives.
  • Data gathering and analysis is moving into the cloud, relieving legacy systems from collection and storage.
  • Data lakes now allow the gathering and synthesis of both structured and unstructured data as it is queried, without the need for local storage.
  • Real-time data streaming technology now allows much faster assessment of incoming data.

So, exactly how does big data help with financial risk management? Here are some examples:

  • It can improve analysis and the quality of models in risk management (e.g. behavior scorecards and bank loan applications).
  • Creditworthiness or fraud risk can be determined in minutes because of information acceleration.
  • Simulations of scenarios based upon huge amounts of data can show concentrations of risks and allow faster reactions to market trends and developments, especially in investment and trading.
  • Big data can provide patterns that indicate potential fraud (e.g. money laundering, credit card fraud, etc.)
  • Applying big data to financial risk management also occurs in algorithm-based trading – something already in substantial use on daily market exchanges.
  • Big data can provide a history and analysis of the types of derivative exposures that have been healthy, and that information can then be used to determine the investment risk posed by similar exposures in a potential investment. Having that data all in one place reduces the time that an investment advisor must spend before making recommendations to clients. Portfolio management becomes an activity with much greater soundness and can be based on the amount of risk exposure a client is willing to take on.

The three biggest advantages in using big data analysis for financial risk management are speed, efficiency, and greater reliability. And the ability to work faster in decision-making gives organizations not just a competitive advantage. Working faster with larger amounts of data means:

  • Predicting credit risks more accurately = better business decisions.
  • Making lending and investment decisions based on longer histories.
  • Getting data in real time to mitigate losses due to fraud and breaches.

Taking Advantage of Financial Risk Management Trends – Examples and Use Cases

Big Data Risk Management

FRM is moving into an “era of data lakes.” All kinds of data are now collected in their actual formats and stored or displayed as such. Both structured data (databases, spreadsheets, etc.) and unstructured data (social media, emails, etc.) can be stored or gathered as a query is made so that real-time information can be accessed.

Here is a couple of general examples of how financial data analytics can be used to mitigate risk:

Perhaps you have concerns about the derivative exposure of a potential investment

Big data can provide a history of similar derivative exposures and a predictive analysis of the amount of risk involved. This means sounder investment strategies and specific investment decisions can be made based on real data rather than just opinions.

You need to perform a credit check on a customer/applicant.

A data lake will let you develop a risk profile based not just upon structured data (e.g. credit report, tax returns) but upon, say, spending habits and social media profiles too. Every fintech startup that engages in lending does this – and decisions are made far faster than the traditional manual collection of only structured data.

Maybe you suspect fraud analyze and stop it early on.

You can use the traditional method of tracking individual trading activities, or you can use a data lake to get a real-time snapshot of all activity, including stuff from chat rooms, mobile devices, even the swiping of entry cards. If you can identify suspicious/fraudulent activity early on, even stop it as it is happening, you are going to avoid fines and a reputation “hit.”

A number of financial enterprises have realized good success in developing and utilizing big data in financial risk management software.

  1. The UOB Bank of Singapore is using big data to test overall risk (value at risk). It can get the analysis within a few minutes. This allows stress tests in real time and much faster response time.
  2. Morgan Stanley is using big data software to optimize its portfolio analysis in terms of both size and quality of results. Because it gets a comprehensive view and automated pattern recognition, it predicts much greater management of financial risk.
  3. Kreditech, a German startup that provides assessments of personal credit-worthiness for its clients, has carved out a great niche, using big data risk management software. They gather both structured and unstructured data, in some instances, from as many as 10,000 data points.
  4. Kabbage, a U.S. company uses a similar approach as Kreditech, to determine creditworthiness of corporate borrowers
  5. Payment analyzes patterns for fraud detection for its credit card company clients. It states that it may analyze multiple millions of transactions in a month for a single client.

How Do You Get on the Big Data Risk Management “Train”?

Advantage of Financial Risk Management

You no doubt have an enterprise data warehouse. And you may already have in-house financial risk management software (or not). Perhaps you are planning to develop risk management software because you have scaled to that point.

Wherever you may be in this process, you will, if you have not already, reach the limits in the amount of data you can gather, store, and analyze. In a recent survey, conducted by Wakefield Research, in fact, 61% of the business decision-making respondents, stated that they are not leveraging data gathering and analysis well enough. Among the reasons, are the following:

  1. They are not able to gather the continually and exploding amount of data that others are using for risk management. And the only way to avoid that is to put the software in place that allows you to gather that big data from those “data lakes” and to get it in real time.
  2. Decision makers are complaining that they cannot get enough information, that information is too slow in coming, and that information is not current enough.
  3. IT executives and staff are complaining that they are struggling to process and analyze new data.
  4. Traditional enterprise data warehouses were not built to gather and analyze exploding big data, certainly now that it is a global activity. And earlier risk management software was not built to do this either. It tends not to gather and analyze unstructured data which has become increasingly important. And what happens is that IT techs have to try to find this data and input it manually, before it can be analyzed. Real-time is lost.
  5. Many decision-makers lament the fact that there is no in-house data scientist, but budgets often preclude such a hire.

You want solutions to all of these challenges, especially if you are looking at what big data can do for your risk management function(s). Specifically, you will want the software that:

  • Can scale to the volume that “data lakes” have;
  • Allows you to gather and analyze structured and unstructured data and do it in real-time;
  • Is cost effective;
  • Ensures security;
  • Can combine, synthesize and analyze collected data all at once and generate reports that are readable and actionable.

To get all of this, you will want the expertise of reputable developers – developers who have experience with both big data and with financial services enterprises, who have a strong reputation, and a history of working with onsite IT operations and finance teams.

Big data for financial risk management is still relatively new. But it is moving fast, and financial services businesses that want to remain competitive and relevant will move quickly too. Whether you begin with just one application or many, you do need to begin.

Romexsoft has had substantial experience in the area of software development in the area of big data financial risk management, whether that is to enhance and upgrade existing software or to develop original software solutions. If you are ready for your first or next big data project, let’s have a discussion.



Hello everyone! I’m Mykhaylo, Managing partner and COO at Romexsoft. I love to track latest software development trends, especially in fintech community. So if you have any questions or need a piece of advice in fintech, feel free to contact me!