We specialise in mathematical data modelling and analysis. The purpose is usually to classify and detect abnormalities.
This is a branch of data-mining often referred to as rare-event or anomaly detection. We have many years experience in this field
and have developed effective techniques to handle this type of hard search Finding a needle in a haystack
The colloquial example of a hard search problem is 'finding a needle in a haystack', however it is even harder
to find a particular grass type in a haystack. The needle is very different and can be easily identified. Suppose,
however, that the hay contains hundreds of different species of grass, and that the task is to identify a particular
rare species of grass that is subtly different from all the others. To make things even more difficult you have
previously only ever seen a few thousand examples of the special grass and this is all that classifications can be
based on.
This contrived example illustrates a type of problem that is common. A typical real-life example is the card
fraud detection problem; here the data consists of millions of transactions which have a multiplicity of features
such that no two are exactly the same. Most are perfectly legal. Only a tiny percentage are known to be fraudulent
and there is almost as much variation amongst these as there is in the legal transactions.
Data like this is so sparse and lacking in consistency, it seems an impossible task to spot future fraud using a
system based on the sample data as the basis of each decision. However, while there simply is not enough information
to go on to make black and white decisions, we can perform powerful screening, in real time, to flag which
transactions should be investigated further by human experts, who would otherwise be overwhelmed by the sheer
quantity of data. The name of the game is to flag likely anomalies/fraud without ‘crying wolf’ too often.
problem.
The data to be analysed typically has the following characteristics:
- Each record has many different attributes (fields).
- There is a logical relationship between records, such as time order.
- A large range of possible variation between records.
- Sparse occurrence of the kind of record we are interested in identifying (a fraction of a percent).
- Insufficient data to make positive IDs from the information contained in the records alone.
- Incomplete data with fields missing
- The information in the fields can often be wrong
Consultancy
We specialise in discovering unseen structures in business data. These can be used to help businesses understand customer behavior,
determine the social structure of their company, detect potential fraud and many other uses.
We can arrange a consultant to discuss how we can help your business gain benefit from its data resource.
Some of the types of analysis services we can provide include:
- Data cleansing and completion
- Analysis of data to discover groups of similar channels/individuals
- Link Analysis to discover groups of cooperating individuals
- Anomaly detection to detect potential fraud
- Relationship discovery - in the context of a supermarket this is often called Basket Analysis
We can examine your data and produce a report on our findings. We can build bespoke systems to address particular needs. We can also deploy a range of in-house technologies and expertise to solve your problem quickly and within budget.
|
