Orchestration of ML-Based Recommendation Systems

Many e-commerce websites use recommendation systems to recommend products to users to boost sales and user experience. These recommendations do not always come from the same recommendation engine. Websites can use multiple recommender models that use different machine learning algorithms and neural networks to compute these recommendations. There arises a need for a machine learning pipeline that will help orchestrate all the steps required to compute and display recommendations. The pipeline handles training a model using content-based approach, storing it with required metadata, loading it, precomputing recommendations, collecting user metrics, analysing the metrics and retraining the models with updated hyperparameters if required. Without a pipeline to automate and streamline the process, much of the work must be done manually.