# MPT-7B: A Revolutionary Leap in Language Models

MosaicML unveiled MPT-7B, a groundbreaking foundational Large Language Model (LLM) for commercial applications.

Previous models were limited by the Llama model's commercial restrictions.

Release note: [Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs](https://www.mosaicml.com/blog/mpt-7b)

**Fine-Tuned Variants:**

1. **MPT-7B-StoryWriter-65k:**
   * Context length of 65,000 tokens, extrapolating up to 84,000 tokens.
   * Demonstrated by crafting an epilogue for The Great Gatsby.
   * Fine-tuned on a dataset of fiction books.
   * ~~Available for commercial use.~~
2. ***MPT-7B-Instruct*****:**
   * Fine-tuned for providing instructions.
   * Utilizes a more extensive dataset than Dolly.
   * Cleared for commercial applications.
3. ***MPT-7B-Chat*****:**
   * Similar to ChatGPT, designed for engaging in dialogues.
   * Commercial usage not permitted due to restricted dataset access.

**Key Takeaways:**

* Pivotal moment in LLM development.
* Training a foundational model is expensive, but fine-tuning is more affordable.
* **Costs:**
  * Base MPT-7B model training cost: over $200,000.
  * MosaicML commendably open-sourced the model.
  * Fine-tuning an instruction-tuned model can cost under $50 (MosaicML did it for $37).
* **Benefits:**
  * Powerful testament to MosaicML platform capabilities.
  * Advantageous for businesses seeking smaller, specialized ChatGPT-like models on their own servers.
  * Addresses potential data privacy concerns.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://ai.saikatkumardey.com/mpt-7b-a-revolutionary-leap-in-language-models.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.