Open-source datasets

• Anthropic Helpful and Harmless (Bai et al., 2022a)
• OpenAI Summarize (Stiennon et al., 2020),
• OpenAI WebGPT (Nakano et al., 2021),
• StackExchange (Lambert et al., 2023)
• Stanford Human Preferences (Ethayarajh et al., 2022)
• Synthetic GPT-J (Havrilla).