“And you're surprised a company, infamous for nerfing it's AIs to the point of being utterly useless in the name of preventing offending someone, has decided to do exactly that one more time why...? Like when will people as a whole realize that the reason they're allowing you to use their model is because they want a dataset of uses of their model that are against and abiding by the rules, so they can nerf the models to output the bullshit you're complaining about. This is literally why they let you use it, to make sure the shit you used it for so far has an example so they can tell the future version of it what NOT to do.
OpenAI is not your friend, they're not giving anyone a free lunch with ChatGPT. They just aren't advertising that they're using all of you to create a better filter against yourselves.”
OpenAI is not your friend, they're not giving anyone a free lunch with ChatGPT. They just aren't advertising that they're using all of you to create a better filter against yourselves.”
👍5
“GPT-2 Output Detector
This directory contains the code for working with the GPT-2 output detector model, obtained by fine-tuning a RoBERTa model with the outputs of the 1.5B-parameter GPT-2 model. For motivations and discussions regarding the release of this detector model, please check out our blog post and report.”
https://github.com/openai/gpt-2-output-dataset/tree/master/detector
https://huggingface.co/openai-detector
This directory contains the code for working with the GPT-2 output detector model, obtained by fine-tuning a RoBERTa model with the outputs of the 1.5B-parameter GPT-2 model. For motivations and discussions regarding the release of this detector model, please check out our blog post and report.”
https://github.com/openai/gpt-2-output-dataset/tree/master/detector
https://huggingface.co/openai-detector
GitHub
gpt-2-output-dataset/detector at master · openai/gpt-2-output-dataset
Dataset of GPT-2 outputs for research in detection, biases, and more - openai/gpt-2-output-dataset
Check out this 3 year old tool trained on GPT-2 data.
Work for you guys?
https://huggingface.co/openai-detector
Work for you guys?
https://huggingface.co/openai-detector
parth007_96’s brilliant notes on reverse-engineering GitHub Copilot:
https://thakkarparth007.github.io/copilot-explorer/posts/copilot-internals
https://thakkarparth007.github.io/copilot-explorer/posts/copilot-internals
GPT-3/LLMs' Achilles heel is short context length - how many "in-context" examples they can consume to learn a new task.
Enter "Structured Prompting": scale your examples from dozens => 1,000+
Here's how:
=> Get 1000s of in-context samples
=> split them into M groups, each small enough to fit in regular context length
=> encode each of M groups using LLM encoder
=> combine these encoded groups and attend over a scaled version of the combination simultaneously
Paper: https://arxiv.org/pdf/2212.06713.pdf
Code: https://github.com/microsoft/LMOps
Enter "Structured Prompting": scale your examples from dozens => 1,000+
Here's how:
=> Get 1000s of in-context samples
=> split them into M groups, each small enough to fit in regular context length
=> encode each of M groups using LLM encoder
=> combine these encoded groups and attend over a scaled version of the combination simultaneously
Paper: https://arxiv.org/pdf/2212.06713.pdf
Code: https://github.com/microsoft/LMOps
👍1