Deep learning methods for plagiarism detection

Plagiarism detection aims to identify instances where an author has copied or closely imitated content from another source without proper attribution. In the digital age, vast amounts of textual data are easily accessible, so plagiarism has become a pressing issue in academia and industry. For instance, a recent survey found that up to 58% of university students admitted to engaging in some form of plagiarism during their academic career, underscoring the scale of the problem. Detecting blatant copy-paste plagiarism is straightforward; however, nuanced plagiarism – such as paraphrasing, synonym substitution, sentence reordering, or idea plagiarism – is much harder to catch. Traditional software often struggles with these subtle cases because it relies on surface-level matching. Early approaches used methods like n-gram overlap, string matching, or simple semantic metrics (cosine or Jaccard similarity) on hand-crafted features. These methods required laborious tuning of similarity thresholds and often failed to catch heavily obfuscated plagiarism.

In plagiarism analysis, there are two main scenarios: extrinsic and intrinsic detection. Extrinsic plagiarism detection (the focus of this article) assumes that potential source materials are available for comparison (for example, checking a suspicious essay against a database or the web) and involves directly comparing the suspicious document to those sources. Intrinsic plagiarism detection, by contrast, attempts to identify plagiarised passages by examining writing style inconsistencies within the suspicious document itself, without needing external references. The deep learning approaches discussed here are primarily applied to extrinsic plagiarism detection, although researchers are also exploring similar neural techniques for intrinsic, style-based detection.

To overcome the limitations of early methods, researchers turned to machine learning – especially deep learning – to automatically learn semantic features and more sophisticated patterns of similarity. Deep learning models can capture the meaning and context of text, making them well-suited for detecting paraphrased or otherwise disguised plagiarism. These neural approaches typically use word embeddings that recognise synonyms (for example, words like “consume” and “eat” end up with similar vectors), so mere word substitutions by a plagiarist are less likely to fool the system. Modern neural networks consider word order and context, so they can identify when two texts express the same idea in different words. As a result, deep learning-based systems significantly outperform earlier methods in detecting paraphrased or otherwise hidden plagiarism.

In the following sections, we provide a detailed look at how CNNs, RNN/LSTMs, and transformer models are used for plagiarism detection. We focus on how these approaches work and how they improve the detection of nuanced plagiarism. (Other machine learning methods – e.g. support vector machines or logistic regression – have also been used for plagiarism detection, but those are beyond our scope here.) Instead, we concentrate on neural network-based techniques capable of capturing complex linguistic patterns and semantic similarities.

Convolutional neural networks for plagiarism detection

Convolutional neural networks (CNNs) are powerful models originally popular in image processing, but researchers have successfully adapted CNNs for natural language tasks, including plagiarism detection. In this context, a CNN treats a sequence of words (or their vector embeddings) as a one-dimensional “signal” and uses convolutional filters to scan for patterns. A CNN slides small filters (e.g. 3-5 words in size) over the text to detect local phrases or n-gram combinations that might indicate copied content. These filters act like feature detectors – for example, a filter might activate on a specific sequence of keywords or a familiar phrase structure. Crucially, CNNs can capture contextual n-grams: if a suspicious document uses many of the same word combinations or phrasing as a source document (even with minor tweaks), the CNN’s filters will pick up those similarities.

After convolution, a pooling layer typically aggregates the strongest signals, reducing dimensionality and helping to prevent overfitting. The model then feeds this output into fully connected layers that produce a similarity score or a plagiarism classification. For instance, a CNN-based plagiarism detector might output the probability that a given pair of texts involve plagiarism (i.e., that one is derived from the other). By scanning for telltale short phrases and partial overlaps, CNNs can find matches that exact string matching would miss – such as when a few words have been changed by the plagiarist but the core expression remains the same. In this way, CNNs help detect even lightly paraphrased plagiarism.

Researchers have employed CNNs both in isolation and as part of siamese network architectures for plagiarism detection. In a siamese setup, two CNN branches process the source and suspicious texts separately to produce vector representations, and the system then compares these representations to judge similarity. Such siamese CNN approaches effectively learn a semantic similarity function between texts. For example, Hambi and Benabbou (2020) developed an online plagiarism detection system that uses a combination of models: it first converts the two input documents into vectors using Doc2Vec embeddings, then a siamese LSTM model determines if the documents are plagiarised, and finally a CNN classifier identifies the type of plagiarism present. This hybrid deep-learning framework achieved about 98% precision in evaluations, substantially outperforming earlier approaches in the educational domain.

This demonstrates that CNNs – especially when hybridised with other models – can contribute to highly accurate plagiarism detectors.

Similarly, other studies have reported strong performance with CNN-based models. For instance, a purely convolutional approach was able to reach over 90% classification accuracy on a benchmark dataset of plagiarised vs. original texts (Agarwal et al., 2018), substantially better than what earlier lexical methods achieved.

One advantage of CNNs is their ability to detect key phrases indicative of plagiarism. Because they operate with fixed-size filters, they excel at identifying local patterns regardless of exact wording. However, a potential limitation is that CNNs alone do not inherently capture long-range dependencies or global context beyond the filter size. They might miss similarities that involve a broader reordering of sentences or extensive use of synonyms across an entire paragraph. Researchers sometimes combine CNNs with sequence-based models or attention mechanisms so that the model considers both local and global textual patterns. For instance, in one study researchers applied a densely connected convolutional network (DenseNet) to the plagiarism data, which delivered high accuracy – although the LSTM model ultimately achieved a slightly higher overall score. This underscores that CNNs are very effective at the lexical pattern level, although incorporating sequence models can capture additional context when needed.

Recurrent neural networks and LSTM models

Recurrent neural networks (RNNs) process text in a sequential manner, preserving word order and accumulating context, which makes them naturally suited for comparing the semantic content of documents. Among RNNs, the Long Short-Term Memory (LSTM) architecture is especially prevalent in plagiarism detection due to its ability to maintain long-term dependencies in text. An LSTM reads a passage word by word (or sentence by sentence), updating an internal state (or “memory”) that effectively “remembers” earlier words. This allows an LSTM to capture the overall meaning of a sentence or paragraph, not just isolated n-grams or keywords.

In the context of plagiarism detection, an LSTM can encode a suspicious document into a vector representation that reflects its semantic content. A second LSTM can do the same for a source document. The system can then compare these representations (for example, by using cosine similarity or a learned neural classifier) to judge whether the documents share the same content. Because LSTMs consider the exact word sequence and context, they are robust when a plagiarist has reworded or rearranged the original text. Even if the specific vocabulary is altered, the sequence of ideas and the contextual flow often remain similar, and an LSTM will pick up on those similarities. Indeed, RNN-based models have shown excellent performance in detecting paraphrases – a closely related task – which directly benefits plagiarism detection on paraphrased text. For example, if the original sentence states “the experiment demonstrated a significant improvement in performance”, and a suspicious sentence reads “the study showed a substantial boost in results”, an LSTM-based model can recognise that both sentences mean the same thing. It will learn that “demonstrated” aligns with “showed”, “significant” corresponds to “substantial”, and “improvement in performance” is equivalent to “boost in results.” A traditional keyword-matching approach might miss this, but the LSTM captures these semantic alignments.

One notable approach is to use a bi-directional LSTM (BiLSTM) in a siamese network configuration. A BiLSTM processes text in both forward and backward directions, capturing context from both sides of each word, which is very useful for fully understanding a sentence. In a siamese BiLSTM model, the suspicious text and the source text each pass through identical BiLSTM encoders, yielding two vector embeddings. The model then compares the resulting embeddings to determine similarity. This architecture can learn to map genuinely similar texts close together in the embedding space, while pushing unrelated texts far apart.

El-Rashidy et al. (2022) implemented a plagiarism detection system that included such an LSTM-based model, and they found that the LSTM approach outperformed a CNN-based approach on standard benchmarks. In their experiments on the PAN 2013 and 2014 plagiarism corpora, the LSTM-based detector achieved the first-ranked performance (highest PlagDet score) compared to up-to-date systems. The researchers attributed the success of the LSTM model to its ability to “weigh” subtle variations in a rich set of textual features, effectively capturing nuanced similarities between sentences. In practice, this means the LSTM could detect plagiarism cases involving significant word substitutions or sentence restructuring that simpler techniques might miss.

Another powerful enhancement for RNN models is the incorporation of attention mechanisms. Attention allows the model to focus on specific parts of the text pair when determining similarity, rather than treating all words as equally important. For example, an attention layer can learn to align particular words or phrases in the suspicious text with their corresponding parts in the source text. This is extremely useful for plagiarism detection because a plagiarised passage often corresponds to some fragment of the source material (possibly in a different order). By highlighting which segments of one text align with segments of the other, an attention-equipped model can more accurately judge if one text is derived from the other. An additional benefit is that attention models can highlight which parts of the suspicious text correspond to which parts of the source, providing a degree of interpretability (e.g., showing the examiner exactly which phrases have essentially been copied or reworded).

Moravvej et al. (2021) proposed an LSTM-based plagiarism detector with an attention mechanism in its architecture. Their model uses two LSTM encoders (for the source and suspicious inputs) and an attention layer that emphasises the most relevant words when comparing the sentence representations. In evaluation, this attentive LSTM model outperformed several baseline methods, demonstrating how attention helps the system spot copied ideas even when the wording is substantially different.

Furthermore, RNN-based systems have been improved with sophisticated training techniques to tackle issues like class imbalance and optimisation difficulties. One challenge in plagiarism datasets is that true plagiarised examples (especially heavily disguised ones) are relatively rare compared to non-plagiarised examples. Researchers have addressed this by modifying how models are trained. For instance, Moravvej et al. (2021) employed a population-based metaheuristic algorithm (an Artificial Bee Colony optimiser) to initialise the LSTM network’s weights instead of using random initialisation. They also applied a special focal loss function to the training objective to handle the class imbalance between plagiarised and non-plagiarised pairs. These techniques ensured the model learned effectively from the minority class (the plagiarised cases) and did not get stuck in poor local optima during training.

As a result of such innovations, LSTM-centric models became more robust and generalised better. The takeaway is that LSTM-based approaches, especially when carefully trained, excel at capturing the sequential and semantic alignment between texts. They effectively “remember” long-range context and can detect when a suspicious text follows the same narrative or logical structure as an original source, even if the wording is different. In summary, RNN/LSTM models bring a powerful ability to model how ideas unfold in writing, which is a critical element in catching disguised plagiarism.

Transformer-based models and BERT

Transformers have revolutionised natural language processing in recent years, and plagiarism detection has also benefited from these advanced models. Transformers use self-attention mechanisms to model the relationships between all words in a text simultaneously, rather than sequentially as RNNs do. This global attention enables them to capture context and meaning with remarkable fidelity. The most prominent example is BERT (Bidirectional Encoder Representations from Transformers) developed by Devlin et al. (2019). BERT is a deep transformer network pre-trained on enormous text corpora (like Wikipedia) to learn language in a general way.

Transformer-based models like BERT provide state-of-the-art semantic understanding: they effectively “read” two texts and determine if they convey the same ideas, regardless of superficial differences in wording or structure. Because of their contextual and bi-directional processing, transformers currently offer the most powerful toolset for detecting nuanced plagiarism.

Plagiarism detection systems have adopted transformer models in several ways. One approach uses a pre-trained transformer like BERT to encode texts and then measures the similarity between the resulting embeddings. Because BERT’s embeddings encapsulate high-level meaning, a plagiarised passage and its original source will yield vectors that are close in the embedding space, whereas unrelated texts will be far apart.

Another approach is to fine-tune a transformer model (such as BERT) directly on a plagiarism classification task. In fine-tuning, the transformer model is further trained on labeled examples of plagiarised vs. non-plagiarised text pairs, and the model learns to output a binary judgment. A fine-tuned BERT has achieved very high accuracy on related tasks like paraphrase identification, and it can reliably detect plagiarism as well. For example, given enough training pairs of original and plagiarised sentences, a BERT-based classifier can learn to identify copied material with both high precision and high recall.

A key strength of transformers is their ability to handle long texts and complex rephrasings. They use multi-head self-attention to find correspondences between words and phrases in one text and those in another text. Suppose a suspicious passage is a heavily paraphrased version of a source paragraph: a transformer model like BERT can still align the key ideas in the suspicious text with the ideas in the source, effectively realising that the two texts are semantically very similar. This kind of deep semantic insight was beyond the reach of older, non-neural methods.

Researchers have reported impressive results by applying transformers to plagiarism detection. Jing and Liu (2023), for example, combined DistilBERT (a lightweight distilled version of BERT) with an LSTM and a differential evolution algorithm to create a high-performance plagiarism detector. DistilBERT is about 40% smaller than the original BERT yet preserves over 97% of BERT’s language comprehension ability, making the model both faster and highly accurate. Their approach outperformed various other deep models and even some earlier heuristic-based algorithms, highlighting how much transformer-derived embeddings can boost detection performance. The use of DistilBERT also shows that even a compressed transformer can capture far more nuance than traditional static word embeddings, thereby detecting plagiarism with greater sensitivity.

Several other studies underscore the impact of transformer models on plagiarism and paraphrase detection. For instance, Laskar et al. (2020) demonstrated that incorporating contextualised embeddings (from BERT and ELMo) into a transformer-based encoder significantly improved sentence similarity modeling in an answer selection task. This finding is relevant because the task of identifying semantically equivalent answers mirrors the challenge of detecting plagiarism between rephrased texts. Likewise, researchers have explored siamese BERT networks, where two BERT models produce embeddings for the two texts which are then compared. Siamese BERT setups allow efficient retrieval of potential source material: for example, one can encode a suspicious document and quickly find the most similar documents in a large collection by comparing embedding vectors. Modern plagiarism detection services can leverage this technique to scan enormous databases for possible sources of a given document.

Notably, the same deep learning methods have also been applied beyond textual plagiarism. Similar neural architectures are being used to detect plagiarism in programming source code (treating code as sequences of tokens) and even in music (using CNN-based models to compare melodic or audio patterns). This underscores the versatility of these approaches in handling various content forms.

Transformer-based models like BERT provide state-of-the-art semantic understanding: they effectively ‘read’ two texts and determine if they convey the same ideas, regardless of superficial differences.

Conclusion

Deep learning-based approaches have transformed plagiarism detection, enabling the identification of copied or rephrased content with unprecedented accuracy. CNN-based models contribute by efficiently spotting local text patterns and phrases that overlap between documents, which helps flag potential plagiarism even if only a few words or an expression match. LSTM-based RNN models add the ability to follow the flow of language and meaning across entire sentences and paragraphs, thereby catching cases where plagiarised content has been paraphrased or reordered rather than directly copied. The inclusion of attention mechanisms further enhances these sequence models by focusing the comparison on corresponding parts of the texts, a feature that is crucial for untangling complex obfuscations.

Transformer-based models like BERT provide state-of-the-art semantic understanding: they effectively “read” two texts and determine if they convey the same ideas, regardless of superficial differences. Because of their contextual and bi-directional processing, transformers currently offer the most powerful toolset for detecting nuanced plagiarism.

Empirical results across numerous studies reinforce these claims. Deep learning models consistently outperform traditional string-matching or heuristic systems, especially on difficult plagiarisms. For example, neural approaches have achieved leading scores in plagiarism detection competitions (such as the PAN workshop evaluations) and have out-ranked earlier methods by substantial margins. In fact, since 2017 the top-ranked systems at the PAN plagiarism detection challenge have almost all employed neural network architectures, underscoring the field’s shift toward deep learning solutions. In one case, an LSTM-based model attained the top PlagDet score on standard benchmarks, and more recent transformer-based models have pushed the performance boundaries even further. These improvements in accuracy mean that paraphrased or cleverly disguised plagiarism, which used to slip through undetected, can now be uncovered reliably.

That said, the success of deep learning does not come without challenges. Training large neural networks requires substantial computational resources and large labeled datasets of plagiarised and non-plagiarised examples. Creating high-quality training data for plagiarism (covering various obfuscation techniques and domains) remains an ongoing effort. Moreover, deep models can be slow to run on very long documents or very large collections. However, researchers are actively addressing these issues. Data augmentation techniques (for example, automatically generating paraphrased copies of texts) can increase training data and improve robustness. Model compression and optimisation methods – such as using DistilBERT instead of full BERT, or distilling ensemble models into single models – are helping to reduce runtime and resource usage. There is also interest in hybrid systems that use fast traditional checks to narrow down candidate matches, then apply deep learning only to promising pairs, thereby combining efficiency with accuracy.

Overall, the combination of CNNs, LSTMs, and transformers provides a comprehensive arsenal for plagiarism detection. CNNs catch the telltale lexical fragments, LSTMs capture the sequential context and syntax, and transformers understand the high-level semantics. Together, these tools allow modern plagiarism detectors to go far beyond surface similarity and actually grasp when two pieces of writing share the same underlying ideas. This leads to robust detection of plagiarism in its many forms – from verbatim copying to heavily obfuscated paraphrasing.

Looking ahead, we expect plagiarism detection to become even more sophisticated as research progresses. Future systems may further hybridise these deep learning models or incorporate new architectures (for example, graph neural networks for analysing citation patterns, or multilingual transformers for cross-language plagiarism) to improve accuracy and scope.

Moreover, the rise of AI-generated writing has introduced new challenges – text produced by large language models (like GPT-4) is not copied from a specific source, but it still violates academic integrity if misrepresented as human work.

Detecting AI-generated text (sometimes considered a form of plagiarism) employs similar techniques, training classifiers to distinguish machine-written content from human writing. We anticipate that plagiarism detection and AI-authorship detection will become increasingly intertwined issues in the near future.

Yet even now, it is evident that deep learning has raised the bar for plagiarism detection:

The fight against plagiarism is thus being bolstered by the latest AI innovations.

In sum, the integration of advanced machine learning techniques has greatly strengthened our ability to uphold academic integrity in the digital era, by making it far harder for plagiarists to escape detection.

References

• El-Rashidy, M. A., Mohamed, R. G., El-Fishawy, N. A., & Shouman, M. A. (2022). Reliable plagiarism detection system based on deep learning approaches. Neural Computing and Applications, 34, 18837–18858. DOI: 10.1007/s00521-022-07486-w
• Hambi, E. M., & Benabbou, F. (2020). A new online plagiarism detection system based on deep learning. International Journal of Advanced Computer Science and Applications, 11(9), 470–478. DOI: 10.14569/IJACSA.2020.0110956
• Moravvej, S. V., Mousavirad, S. J., Moghadam, M. H., & Saadatmand, M. (2021). An LSTM-based plagiarism detection via attention mechanism and a population-based approach for pre-training parameters with imbalanced classes. In Proceedings of the 28th International Conference on Neural Information Processing (ICONIP 2021), Lecture Notes in Computer Science, vol. 13109, pp. 690–701. Springer.
• Jing, Y., & Liu, Y. (2023). Population-based plagiarism detection using DistilBERT-generated word embedding. International Journal of Advanced Computer Science and Applications, 14(8), 618–626. DOI: 10.14569/IJACSA.2023.0140872
• Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT 2019, pp. 4171–4186.
• Bao, W., Du, J., Yang, Y., & Zhao, X. (2018). Attentive Siamese LSTM network for semantic textual similarity. In Proceedings of the 2018 International Conference on Asian Language Processing (IALP), pp. 312–317. IEEE.
• Laskar, M. T. R., Huang, X., & Hoque, E. (2020). Contextualized embeddings based transformer encoder for sentence similarity modeling in answer selection. In Proceedings of the 12th Language Resources and Evaluation Conference (LREC 2020), pp. 5505–5514.
• Shakeel, M. H., Karim, A., & Khan, I. (2020). A multi-cascaded model with data augmentation for enhanced paraphrase detection in short texts. Information Processing & Management, 57(3), 102204. DOI: 10.1016/j.ipm.2020.102204