FinRL_DeepSeek

FinRL_DeepSeek

Code for the paper "FinRL-DeepSeek: LLM-Infused Risk-Sensitive Reinforcement Learning for Trading Agents" arXiv:2502.07393

Stars: 100

Visit
 screenshot

FinRL-DeepSeek is a project focusing on LLM-infused risk-sensitive reinforcement learning for trading agents. It provides a framework for training and evaluating trading agents in different market conditions using deep reinforcement learning techniques. The project integrates sentiment analysis and risk assessment to enhance trading strategies in both bull and bear markets. Users can preprocess financial news data, add LLM signals, and train agent-ready datasets for PPO and CPPO algorithms. The project offers specific training and evaluation environments for different agent configurations, along with detailed instructions for installation and usage.

README:

FinRL-DeepSeek: LLM-Infused Risk-Sensitive Reinforcement Learning for Trading Agents

Blog: https://melwy.com/finrl_deepseek

Paper: https://arxiv.org/abs/2502.07393

Update1: The project is integrated to the original FinRL project by AI4Finance!

Update2: The project is the basis of task 1 in FinRL contest 2025!

Installation script: installation_script.sh

Data: https://huggingface.co/datasets/benstaf/nasdaq_2013_2023/tree/main

Trading agents: https://huggingface.co/benstaf/Trading_agents/tree/main

Backtesting Notebook: Open In Colab

Results

Alt Text

Preliminary conclusion

Bull market -> PPO

Bear market -> CPPO-DeepSeek

More details on installation of dependencies

run installation_script.sh on Ubuntu server (128 GB RAM CPU instance recommended)

Datasets and data preprocessing

The basic dataset is FNSPID: https://huggingface.co/datasets/Zihan1004/FNSPID (the relevant file is Stock_news/nasdaq_exteral_data.csv)

https://github.com/Zdong104/FNSPID_Financial_News_Dataset

https://arxiv.org/abs/2402.06698

LLM signals are added by running sentiment_deepseek_deepinfra.py and risk_deepseek_deepinfra.py, to obtain:

Then this data is processed by train_trade_data_deepseek_sentiment.py and train_trade_data_deepseek_risk.py to generate agent-ready datasets.
For plain PPO and CPPO, train_trade_data.py is used.

Training and Environments

  • For training PPO, run:
    nohup mpirun --allow-run-as-root -np 8 python train_ppo.py > output_ppo.log 2>&1 &

  • For CPPO: train_cppo.py

  • For PPO-DeepSeek: train_ppo_llm.py

  • For CPPO-DeepSeek: train_cppo_llm_risk.py

Environment files are:

  • env_stocktrading.py for PPO and CPPO, same as in the original FinRL
  • env_stocktrading_llm.py or env_stocktrading_llm_01.py for PPO-DeepSeek (depending on the desired LLM influence. More tweaking would be interesting)
  • env_stocktrading_llm_risk.py or env_stocktrading_llm_risk_01.py for CPPO-DeepSeek

Log files are output_ppo.log, etc., and should be monitored during training, especially:

  • AverageEpRet
  • KL
  • ClipFrac

Evaluation

Evaluation in the trading phase (2019-2023) happens in the FinRL_DeepSeek_backtest.ipynb Colab notebook.
Metrics used are Information Ratio, CVaR, and Rachev Ratio, but adding others like Outperformance frequency would be nice.

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for FinRL_DeepSeek

Similar Open Source Tools

For similar tasks

For similar jobs