In 2019, FloQast released a product called AutoRec, which automates the account reconciliation process, saving accountants hours or days they would otherwise spend manually sifting through transactions each month to identify true exceptions or reconciling items. This is often achieved by comparing two data sets, such as the general ledger against the bank statement.

In order to accomplish that task, we developed a matching engine that used numerous algorithms to comb through the data and automatically match transactions on behalf of the user. But, as the number of AutoRec users increased, so did the number of variations in how matches are made for different clients. This led to a growing number of configuration options and increased complexity in the engine, as well as a growing number of custom configurations that had to be maintained for individual clients. We needed a more scalable solution!

So, we asked ourselves, can we use machine learning here? At the time, I was new to the engine team and had no experience with machine learning. I had more questions than answers.

Even today, there are a number of questions about machine learning (and quite a few misnomers, as well) and its capabilities. So, I’ll attempt to help you build a basic intuition for what machine learning is and how it works.

The term “machine learning” has become immensely popular in the last few years, but it’s not always clear what exactly is being “learned” and how. And while this is a vast field of study with many different techniques, algorithms, and applications, we’ll use a simple example to illustrate the core concept which is at the heart of machine learning.

Let’s begin with how we humans may learn from data that we’re presented with. Suppose you were given the following data and asked to figure out what the values for A and B might be:

(A x 2) + ( B x 10) = 30

(A x 3) + ( B x 20) = 55

(A x 5) + ( B x 30) = 85

You probably correctly arrived at the conclusion that A = 5 and B = 2 [Editor’s Note: It was my understanding that there would be no math…]. Now, think about the process that took place in your mind as you were solving the problem. It may have looked something like this:

1. Take a guess at initial values for A and B
2. Calculate the result for each line using the selected values
3. Check the results against the given answers
4. Adjust values for A and B and repeat until the results match the answers (as best as possible)

As it turns out, there are algorithms that know just how to do that! In fact, machine learning is essentially a set of established algorithms that make learning possible. One such algorithm is called gradient descent. The initial weights may start with zeros or random values. The more interesting part is how it adjusts the weights to get closer to the correct answers by minimizing the cost (the difference between the current results and the given answers) with each iteration. It does so using formulas based on derivatives to determine whether each weight should be increased or decreased.

Now that we learned the optimal weights based on the given input data (also known as training data), we have a model (Read: Set of weights) that can be used to predict the output for new data for which we don’t have the answers.

You may also have a binary classification problem in which the training answers are either 1 (yes) or 0 (no) and the model will be trained to predict a value (also known as a probability) between 0 and 1, in which case you might consider a probability of 0.5 or higher to be a yes prediction.

If we step back and look at what machine learning is doing, it’s learning the relative importance (weight) of the individual features in relation to the output being predicted.

So for the matching engine, we realized that if we could come up with a way (i.e. features) to represent what a matching pair of transactions looks like, we can use machine learning to train a model that can predict if we have a match. And better yet: We don’t have to use custom configuration to account for the variations across FloQast users because the model can learn these variations all on its own, and we end up with virtually a zero-configuration matching engine that doesn’t have to increase in complexity as more clients use it.

It also makes life easier for our clients and our support team because they no longer have to look at the data and try to come up with their own set of matching rules, which must be designed and maintained by the user, to produce the best matches.