What is wrong with ChatGPT-generated code

There are a few basic things that ChatGPT gets wrong:

  1. ChatGPT hallucinates often - it produces function names that don't exist
  2. ChatGPT does not understand that AFL is case-insensitive, and because of this it generates variable names conflicting with built-in function names, for example:

    // "RSI" is a built-in function
    rsi = RSI( 14 ); // WRONG "rsi" can't be used for variable name
  3. ChatGPT does not understand inner workings of AmiBroker, for example it does not know how backtesting works, what a custom backtester is, and such, so it produces code that might look OK and even compiles without errors, yet it is nonsense.

If you have time I strongly recommend reading the entire thread on the forum covering ChatGPT and AFL:
https://forum.amibroker.com/t/chatgpt-is-utterly-amazing-and-scary-at-the-same-time/33212/1

(As an extra bonus - the text below was generated by ChatGPT itself, in response to "Explain to a casual reader why using ChatGPT-generated AFL code is a bad idea" )

Using ChatGPT-generated code, especially for complex tasks like creating code for the AFL (AmiBroker Formula Language), is generally a bad idea for a few reasons:
  1. Lack of Understanding: ChatGPT generates code based on patterns it has learned from a wide range of examples. However, it doesn't truly understand the logic or purpose behind the code. This means that the code it generates might work in some cases but could fail miserably in others. It doesn't have the ability to comprehend the intricacies of the AFL language, trading strategies, or market behavior.
  2. Unreliable and Unsafe: AFL code generated by ChatGPT might not be reliable or safe for actual trading. Trading decisions can have significant financial implications, and relying on code generated by an AI that isn't equipped to understand market dynamics could lead to substantial losses.
  3. Lack of Context: ChatGPT doesn't have real-time market data or awareness of the current market conditions. It can't adapt to changing situations or trends, which is crucial for trading strategies.
  4. Over-Optimization: One common mistake in algorithmic trading is over-optimizing strategies based on historical data, which can lead to poor performance in real-world trading. ChatGPT-generated code might inadvertently encourage overfitting to past data without considering its generalization to new data.
  5. Regulatory Concerns: Depending on your jurisdiction, there might be legal and regulatory considerations when it comes to algorithmic trading. Relying on code generated by an AI might lead to unintentional violations of trading rules or regulations.
  6. Lack of Testing and Validation: Code generated by ChatGPT hasn't undergone proper testing, debugging, or validation processes that are essential for creating robust and effective trading strategies.
  7. Complexity of Trading Strategies: Trading strategies involve a deep understanding of financial markets, technical analysis, risk management, and more. Creating successful trading strategies goes beyond just writing code; it requires domain expertise that ChatGPT lacks.
  8. Missed Opportunities: Relying on AI-generated code might lead to missing out on valuable learning opportunities. Developing your own code and strategies allows you to learn from mistakes and continuously improve your skills.

In summary, while ChatGPT is a powerful tool for generating text, it's not a substitute for human expertise and understanding in complex domains like trading and coding. If you're interested in algorithmic trading, it's essential to invest the time to learn the fundamentals, acquire the necessary domain knowledge, and develop your strategies with careful consideration and testing.