Country Financial investment selection using LLM based SQL Generation and RAG

Country Financial investment selection using LLM based SQL
Generation and RAG

The Challenge

A financial research platform sought to develop a solution for processing large-scale financial reports and economic documents to forecast various country specific macro economic factors, along with conducting sentiment analysis on the forecasts. Additionally, they required the development of a chatbot capable of handling natural language queries from users and generating responses by retrieving data from a structured database. Another key objective was to create a financial knowledge base derived from both structured and unstructured text information.

Hypothesis

  • The solution requires the development of a predictive AI capable of forecasting economic trends and financial data. A classifier must be implemented to perform sentiment analysis on these forecasts.
  • Additionally, an intelligent data processing pipeline needs to be developed to convert natural language queries into SQL, which will retrieve structured information from the database.
  • Furthermore, a data pipeline will be built to populate the knowledge base, serving as a key source of information for generating answers to user queries.

Execution

  • A pipeline leveraging NLP and deep learning techniques was designed to predict and categorize financial forecasts from documents.
  • A custom large language model (LLM) application was employed to convert natural language queries into SQL, with provisions to account for variations in user vocabulary.
  • Furthermore, an intelligent question prefetching system was developed to anticipate and answer similar queries with minimal delay, optimizing response time and efficiency.

Outcomes

  • An optimized knowledge base was created by merging structured and unstructured data from extensive macroeconomic documents.
  • A chatbot was also developed to handle natural language queries, converting them into SQL commands to extract structured data from the database.
  • Additionally, a forecasting system with sentiment analysis was implemented to deliver meaningful financial insights to users.

Project Highlights

  • Massive increase in accessibility and accuracy of financial planning and value prediction