Forwarded from Chat GPT
Can AI-Generated Text be Reliably Detected? — NO, if you (1) employ paraphrasing and (2) if your LLM isn’t being used to spy on you.
Empirically, we show that paraphrasing attacks, where a light paraphraser is applied on top of the generative text model, can break a whole range of detectors, including the ones using the watermarking schemes as well as neural network-based detectors and zero-shot classifiers.
We then provide a theoretical impossibility result indicating that for a sufficiently good language model, even the best-possible detector can only perform marginally better than a random classifier.
We use a PEGASUS-based [Zhang et al., 2019] paraphraser to rephrase this watermarked output from the target LLM. The paraphraser rephrases sentence by sentence. The detector does not detect the output text from the paraphraser. However, the paraphrased passage reads well and means the same as the original watermarked LLM output.
PEGASUS paraphraser.
Paper.
Empirically, we show that paraphrasing attacks, where a light paraphraser is applied on top of the generative text model, can break a whole range of detectors, including the ones using the watermarking schemes as well as neural network-based detectors and zero-shot classifiers.
We then provide a theoretical impossibility result indicating that for a sufficiently good language model, even the best-possible detector can only perform marginally better than a random classifier.
We use a PEGASUS-based [Zhang et al., 2019] paraphraser to rephrase this watermarked output from the target LLM. The paraphraser rephrases sentence by sentence. The detector does not detect the output text from the paraphraser. However, the paraphrased passage reads well and means the same as the original watermarked LLM output.
PEGASUS paraphraser.
Paper.
👏8❤1👍1🤔1
Forwarded from Chat GPT
Are AI writing detectors actually just linguistic stupidity detectors? (Yes)
GPT detectors are biased against non-native English writers.
Our findings reveal that these detectors consistently misclassify non-native English writing samples as AI-generated. This study reveals a notable bias in GPT detectors against non-native English writers, as evidenced by the high misclassification rate of non-native-authored TOEFL essays, in stark contrast to the near zero misclassification rate of college essays, which are presumably authored by native speakers. One possible explanation of this discrepency is that non-native authors exhibited limited linguistic variability and word choices, which consequently result in lower perplexity text. Non-native English writers have been shown to exhibit reduced linguistic variability in terms of lexical richness [24], lexical diversity [25], [26], syntactic
complexity [27–29], and grammatical complexity [30].
Paper
Github
GPT detectors are biased against non-native English writers.
Our findings reveal that these detectors consistently misclassify non-native English writing samples as AI-generated. This study reveals a notable bias in GPT detectors against non-native English writers, as evidenced by the high misclassification rate of non-native-authored TOEFL essays, in stark contrast to the near zero misclassification rate of college essays, which are presumably authored by native speakers. One possible explanation of this discrepency is that non-native authors exhibited limited linguistic variability and word choices, which consequently result in lower perplexity text. Non-native English writers have been shown to exhibit reduced linguistic variability in terms of lexical richness [24], lexical diversity [25], [26], syntactic
complexity [27–29], and grammatical complexity [30].
Paper
Github
🤣12👍3❤1😁1
NaturalSpeech 2: Latent Diffusion Models are Natural
and Zero-Shot Speech and Singing Synthesizers
New AI can generate singing from only a speech prompt input.
Demo Website
and Zero-Shot Speech and Singing Synthesizers
New AI can generate singing from only a speech prompt input.
Demo Website
🔥4❤1
AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models
AI that can follow your audio editing instructions.
Demo Website
AI that can follow your audio editing instructions.
Demo Website
👍8🔥3❤1