7.4 GPT-3.5, GPT-4, ChatGPT and Bard

LLMs - GPT-3.5, GPT-4, ChatGPT and Bard

GPT-3.5 and GPT-4 are examples of Large Language Models (LLMs) that are based on a Transformer architecture. GPT-3.5 was developed by OpenAI, and is one of the largest and most powerful LLMs currently available, recently superceded by GPT-4. 

GPT-3.5 and GPT-4 uses the Transformer attention mechanism to selectively focus on different parts of the input sequence during processing, and enables the Transformer model to capture long-range dependencies between words in a sentence, needed for natural language processing tasks such as language modeling and text generation.

Bard is a LLM publicly released by Google in May 2023 that generates human-like responses. Bard is user-friendly, with a simple interface and importantly also provides realtime access to the internet and is likely to contain information that is being continually indexed by Google web crawlers.

Trained with nearly all Human information on the Internet

In terms of the number of words, it's difficult to give an exact figure since the training information used to train ChatGPT, based on GPT-3.5, consists of a vast body of text from the Internet up to Sep 2021, including websites, books, and articles. However, it's estimated that the training information used to train GPT-3.5 consists of several terabytes of text information, which likely includes hundreds of billions of words. The exact number of words and parameters used in ChatGPT can vary depending on the specific configuration and version of the LLM. In the ChatGPT system which uses GPT-3.5, it has a total of 175 billion parameters, and the latest version, GPT-4, uses 1,000 billion parameters [143].

ChatGPT is a generative model (using either GPT-3.5 or GPT-4 depending on user subscription), which means that it is Capable of generating text based on a given input prompt or context. For example, if you prompt ChatGPT with: "What is the capital of France?", it might generate the response: "The capital of France is Paris." The quality of the generated response depends on the size and quality of the training information that ChatGPT contains, as well as the specific configuration of the model.

It's worth noting that the size and complexity of LLMs like GPT-3.5 and GPT-4 can have a significant impact on their performance and Capabilities. The bigger LLMs are generally better at capturing more complex relationships within the input information, but can also be more computationally intensive and require more resources and time to train and run.

The recent early 2023 LLMs and the Transformer architecture represent a very significant advancement in the development of more sophisticated and highly interactive Human-like AI language models.