THE EVOLUTION OF TRANSACTION COST ANALYSIS.

By Dr Mainak Sarkar, Head of European Execution Advisory, and James Baugh, Head of European Equities Market Structure, Citi.

The post trade workflow for most institutions on the buy side typically consists of the following steps: record all trade data and enrich it with the relevant market data in order to compute a set of diverse benchmarks, the most popular being Implementation Shortfall (IS), volume weighted average price (VWAP) and participation-weighted-price (PWP). The performance of orders measured against these benchmarks is then analysed based on stock and order level characteristics (such as the bid-offer spread, size of the order as percentage of daily volume, daily volatility of the stock and so on), this is the attribution phase. Finally the performance data is usually analysed by broker to measure any systematic differences in execution, all of this is as mandated under current regulations.

Limitations

We start by noting the challenges associated with each step of the post trade process:

• Data quality issues and latency in data capture during the measurement phase; we observe considerable diversity in data infrastructure and investment amongst buy-side firms that connect to Citi as a broker. Most institutions do not maintain their own market data as it involves significant cost of setup and maintenance and usually resort to buying the same from third party providers, this can lead to an added layer of complexity and challenge. We do encounter frequent complaints by clients with regards to the quality of the data received from vendors which can lead to inconsistent estimation of performance.
• Selecting the right benchmarks is critical and this can vary by the type of firm and trading objectives; for instance a quant fund with short term alpha compared to passive investors; have both different trading objectives and horizons and therefore utilise different performance benchmarks. A narrow focus on a particular benchmark can potentially lead to the algorithm trying to achieve that goal possibly at the exclusion of other important goals thereby hampering best execution, as broadly defined under MiFID II. For instance slippage against arrival may be reduced by trading intensively (small orders only) however this can lead to high levels of impact and price reversion, which is not desirable. Conversely too many objectives can often cloud the findings regarding performance therefore making any changes to trading difficult. As a broker we have observed significant variation in how clients approach this and there is no single size fits all solution to this problem and various clients are grappling with this on an ongoing basis.
• Attribution can often be challenging- for instance if the stock moved adversely during trading, decomposing this move into market impact caused by the trading of the algorithm with the rest being natural alpha in the stock is usually challenging, various approaches using index and sector level beta adjustment exist however they are not entirely satisfactory in their outcome. Similarly small sample sizes can lead to the analysis frequently being restricted (or dominated by notional weighting) to the large capitalisation / more liquid names with significant order flow, however the performance across brokers in the less liquid category may be more relevant but difficult to measure. .
• Evaluation of brokers may be similarly biased due to the skewed distribution of order flow in terms of difficulty particularly when they are manually traded, this may be driven by long time perceptions of particular strategies and brokers. The algorithmic wheel setup seeks to ameliorate these problems by randomising the flow of orders across brokers. Citi’s experience in this regard has been varied as we are well represented on multiple wheels across a host of clients, wheels are frequently time consuming to setup and calibrate as the right data is collected across brokers and adjusted for order differences to achieve a fair comparison, this process can be quite challenging.
• Statistics / machine learning: working with large datasets (larger clients can trade thousands of orders in a day) usually requires key decisions to be taken regarding how to deal with outliers (identify, remove, winsorise, etc), how to price the unfilled portion of any orders (opportunity cost), whether to penalise under participation by brokers under adverse conditions (scaling wrongly), etc. mostly in an automated fashion, all of which require sophisticated analysis on a large enough sample of orders by specialised team to draw meaningful conclusions. Citi has observed a significant uptick in the hiring of such professionals in recent times on the buy side. There is a trend towards using more advanced statistical analysis and machine learning techniques to analyse performance. This is an industry trend which has been strengthened by regulatory requirements.

Citi recognised the complexity of the above mentioned process of post trade analytics and created an Execution Advisory team, whose role was to engage and consult with clients to help improve their benchmark performances. There has been significant engagement across a range of clients who have benefitted from this more consultative approach.

Challenges for the broker

A significant development in the post MiFID world has been the increasing automation of the trader workflow and has seen the growth in the use of algorithmic wheels. The wheel engine randomly identifies and allocates a set of ‘easy to trade’ orders across a list of brokers. The identification of ‘easy to trade’ can be either done via a pre trade model (estimated impact cost function) or by a set of rules such as percentage of daily volume and/or notional limits (all orders less than $1 million are routed to the wheel for instance). At set frequencies – usually a month or a quarter – the performance across brokers is analysed and the wheel is reweighted such that the broker performing better receives more orders from the client over the next period. Note that this can lead to an imbalanced sample in the next period and therefore statistically difficult comparisons across brokers, as some brokers have large samples and others significantly smaller ones. Having too many brokers on the wheel can potentially lead to orders being thinly spread and therefore sample sizes being too small, the optimal number of brokers is therefore client and flow specific. Finally for fair comparison ideally a post trade normalisation of order performance across brokers is desirable, for instance if Broker A received predominantly certain orders on news or event days when volumes and volatility are usually high; this needs to be factored into the analysis while evaluating brokers.

Brokers are continuously in the process of upgrading their algorithms based on internal analysis as well as feedback from clients, however such redesigns are often time consuming and expensive in terms of resources. Therefore fundamental changes in the trading platform occur only infrequently and usually most innovations are incremental in nature. Therefore short term changes to the strategies are unlikely to drive large fluctuations in execution outcomes for clients, given that significant care is usually taken by the broker to roll out only changes which are well tested and are expected to improve the outcomes on average. The volatility observed in rankings can often be greater than the infrequent changes to algorithms which leads one to suspect that the evaluation mechanism on the buy side can be improved by filtering out the noise in the data better leading to more stable comparisons, from a broker perspective.

For medium to small size firms infrastructure is often restricted to consuming TCA provided by the broker themselves. Assuming complete impartiality it is observed that due to significant differences in definitions, data capture and storage issues, and measurement errors, performance measurement can vary widely across brokers making comparison problematic for the buy-side. The subjective element of measurement is significant when comparing TCA across brokers.

In our experience even for those larger firms that are able to capture their own data, the numbers invariably do not match the brokers’ numbers and a significant amount of time and effort is subsequently required to reach some level of agreement and consensus, which is essential to ensure that the data is sufficiently clean before drawing any conclusions from it which will drive the execution outcomes.

Whilst we have seen an increased interest in independent providers of TCA, which can apply a more uniform and consistent set of measures (benchmarks) against the data, they are not immune to these same challenges. Moreover, they lack the understanding and the nuances of broker specific execution strategies. Applying a one size fits all approach to data across brokers loses out on the broker specific differences that are key to achieving best execution. Without the right context as to why the algorithm behaved in a particular way the filtering of orders is based on rules which may not give informative outcomes.

Innovations at Citi

Citi is developing the use of advanced machine learning techniques to drive a more proactive approach to the TCA process. A number of these techniques, including the use of peer group analysis and clustering, are discussed in this article.

1. Peer analysis

A new feature selection approach using Random Forest models has been implemented to highlight how different variables contribute to the overall performance of flow from a particular underlying client. This tool is also helping to identify areas of underperformance via potential trader biases under different market conditions when the price may be moving in and out of favour, such as systematic modifications, limit price selections, cancellations, etc.

Actionable TCA of this nature is key in delivering client specific recommendations, which in turn is leading to better overall performance.

Machine learning based Peer Analysis has also been developed at Citi. Clusters are compared across different clients with similar objectives thereby identifying any systematic differences in order flow. Using clusters identified for each client, we can make recommendations on customisations and on strategies to optimise client performance. Exhibit 1 illustrates this approach.

1. Stock categorisation

Citi is also using machine learning methods to identify different clusters of orders. We use various stock level characteristics. The most standard being bid-offer spread, volatility and queue sizes. Other more microstructure level features are used to endogenously categorise each stock in the universe to a particular category. For instance one category may be high volatility, small queue size stocks. There are multiple applications of this approach both at the pre-trade and also post-trade phase.

This model can be used pre trade to determine the right strategy. For instance where a client is using VWAP algorithms for low volatility but wide spread stocks. The EAS team actively monitors the performance goals and can use these tools to come up with custom recommendations for different clients. In this case one recommendation may be to change the passive layering of the book. In essence security and order level clustering methods allow us to draw conclusions at the client level which ultimately leads to better execution performance.

III. Forward looking

Citi is enhancing its current historic data and analytics based approach by incorporating a forward looking simulation methodology, through the provision of ‘scorecards’. Citi believes that this hybrid method leads better insight into performance measurement. Each scorecard highlights alternative strategies for liquidity provisioning. This innovative approach marries the client’s objective with market conditions and available internal liquidity to come up with theoretical improvements in execution versus the client’s benchmark.

The scorecard can be used by high or low touch trading desks and or central risk to adjust for commissions and make client specific recommendations depending on what the optimal strategy is for them to execute their flow.

Recommendations can be tailor made for clients with different flows. Exhibit 2 illustrates a sample scorecard approach, which shows cost savings / slippage versus commission rates for liquidity sourcing.

Optimising for TCA performance has been a key focus of the research and development here at Citi. Extensive study and back-testing has revealed that the price and size alone should not be the only consideration when interacting with different liquidity sources. Citi has employed machine-learning algorithms to monitor intra-trade market activity and real time order performance to drive its algorithm behaviour. Real-time venue ranking also plays a crucial part in sourcing not only the best price and size, but also takes into account the quality of the liquidity. In regards to interacting with Systematic Internalisers, Citi’s comprehensive real time venue analytics measures different characteristics such as short term toxicity, uniqueness of liquidity, reversion, good and bad fills, fill rate and quote and fill toxicity. For example, we not only look at the child-level fill performance to measure the quality of our fills, but prior to the child trade we establish the toxicity of the quote from each of the Systematic Internalisers.

This information is fed to the Smart Order Router to determine its routing decisions. Furthermore, the routing decision for taking and posting liquidity is also regulated for different strategy types based on the urgency level – a very aggressive strategy may still interact with toxic liquidity sources to satisfy liquidity demand. These enhancements help us to tailor our execution strategies and improve performance.

1. Case study

TCA driven enhancements to algorithms

Citi measures performance across a wide range of clients and uses the findings in designing better products that help clients achieve optimal outcomes – an example is given below. Citi is in the process of rolling out significant enhancements to Dagger, its flagship liquidity-seeking algorithm (LSA) for equities. Earlier TCA reports gave a starting point of where the algorithm could be calibrated more effectively. This was combined with more research by the quantitative team to build a new algorithm which performs better against a wide spectrum of market conditions and stocks. Performance was evaluated using a standard A/B race on Citi principal order flow from Feb 2019 to May 2019. Trades numbered over ten thousand, with a value of over $1bn. Results showed significant performance improvements. Some of the key benefits were:

• Spread capture improved by 54%, with similar average fill rates;
• Spread crossing reduced by 42%;
• Similar average order duration and fill rates to the classic Dagger.

Key performance drivers were:

• Move away from heuristic based, reactive logic to a continuous optimal trajectory model based on machine learning analytics;
• Improved strategy sensitivity to possible adverse selection scenarios;
• Use real time analytics in addition to historical to rank venues;
• Machine learning algorithm for dynamically choosing passive allocation policy based on real time performance.

Performance was analysed on different aspects/slices of the data in terms of order as percentage of ADV, volatility, spread, market move and participation rates. The most significant improvement was observed in the category where the prices ‘come in’. Exhibit 3 shows the distribution of the slippage for this category; note the significant improvement in performance. We also note that the performance remains consistent when the prices are moving away. Furthermore, the improved Dagger is more capable of sourcing liquidity passively and at mid. This is an example of where machine-learning components in the algorithm have helped to provide a better performance to the end client.

Conclusion

In this note we have outlined some of the shortcomings of the post trade analytics process (of which TCA is an integral part), that in our experience clients have encountered frequently. Some of which can be fixed via more and better investments in infrastructure (better data quality) however others are more intractable (normalisation of orders by broker for instance) and require continuing research and analysis by our buy-side colleagues. The Execution Advisory team here at Citi is a resource to assist with this process in every step of the way.

A number of innovations undertaken in this space have been outlined above. Our application of advanced machine learning techniques to TCA and using real-time analytics to minimise the implicit costs of trading is proving invaluable in providing the performance metrics our clients need to improve performance, meet best execution requirements and measure and rank broker performance.

Finally we gave an illustration of product design based on inputs obtained from TCA which led to significant improvement for the client and better execution outcomes through the use of advanced machine learning techniques, calibrated using the findings from TCA.

For further information, please contact: emea.eas@citi.com

©BestExecution 2020

[divider_to_top]