Quick start

ollama run mistrallite:latest

Runnable variants

Memory is a rough file-size estimate; context length, runtime and GPU offload change it.

12 of 17 matching variants shown.17 of 17 variants shown.

Tag	Type	Inputs	Digest	Download	Parameters	Quantization	Context	Est. memory	Run
7b	Typelocal	Inputstext	Digest5393d4f5f262	Download4.1 GB	Parameters7b	QuantizationUnknown	Context32K	Est. memory≈ 6 GB	`ollama run mistrallite:7b`
7b-v0.1-fp16	Typelocal	Inputstext	Digesta6c59d49e2a9	Download14 GB	Parameters7b	QuantizationFP16	Context32K	Est. memory≈ 17 GB	`ollama run mistrallite:7b-v0.1-fp16`
7b-v0.1-q2_K	Typelocal	Inputstext	Digest32164f7b30a5	Download3.1 GB	Parameters7b	QuantizationQ2_K	Context32K	Est. memory≈ 4.5 GB	`ollama run mistrallite:7b-v0.1-q2_K`
7b-v0.1-q3_K_L	Typelocal	Inputstext	Digestbb7562f0532f	Download3.8 GB	Parameters7b	QuantizationQ3_K_L	Context32K	Est. memory≈ 5.5 GB	`ollama run mistrallite:7b-v0.1-q3_K_L`
7b-v0.1-q3_K_M	Typelocal	Inputstext	Digest366ee513d776	Download3.5 GB	Parameters7b	QuantizationQ3_K_M	Context32K	Est. memory≈ 5 GB	`ollama run mistrallite:7b-v0.1-q3_K_M`
7b-v0.1-q3_K_S	Typelocal	Inputstext	Digestebf457323d7d	Download3.2 GB	Parameters7b	QuantizationQ3_K_S	Context32K	Est. memory≈ 5 GB	`ollama run mistrallite:7b-v0.1-q3_K_S`
7b-v0.1-q4_0	Typelocal	Inputstext	Digest5393d4f5f262	Download4.1 GB	Parameters7b	QuantizationQ4_0	Context32K	Est. memory≈ 6 GB	`ollama run mistrallite:7b-v0.1-q4_0`
7b-v0.1-q4_1	Typelocal	Inputstext	Digest00a4a71eee8f	Download4.6 GB	Parameters7b	QuantizationQ4_1	Context32K	Est. memory≈ 6.5 GB	`ollama run mistrallite:7b-v0.1-q4_1`
7b-v0.1-q4_K_M	Typelocal	Inputstext	Digestd73bbd6b49d1	Download4.4 GB	Parameters7b	QuantizationQ4_K_M	Context32K	Est. memory≈ 6 GB	`ollama run mistrallite:7b-v0.1-q4_K_M`
7b-v0.1-q4_K_S	Typelocal	Inputstext	Digest81a8a1744387	Download4.1 GB	Parameters7b	QuantizationQ4_K_S	Context32K	Est. memory≈ 6 GB	`ollama run mistrallite:7b-v0.1-q4_K_S`
7b-v0.1-q5_0	Typelocal	Inputstext	Digest751a158ec222	Download5.0 GB	Parameters7b	QuantizationQ5_0	Context32K	Est. memory≈ 7 GB	`ollama run mistrallite:7b-v0.1-q5_0`
7b-v0.1-q5_1	Typelocal	Inputstext	Digeste09d24104dd5	Download5.4 GB	Parameters7b	QuantizationQ5_1	Context32K	Est. memory≈ 7.5 GB	`ollama run mistrallite:7b-v0.1-q5_1`
7b-v0.1-q5_K_M	Typelocal	Inputstext	Digest7a3f8c87136f	Download5.1 GB	Parameters7b	QuantizationQ5_K_M	Context32K	Est. memory≈ 7 GB	`ollama run mistrallite:7b-v0.1-q5_K_M`
7b-v0.1-q5_K_S	Typelocal	Inputstext	Digest6b71023e7195	Download5.0 GB	Parameters7b	QuantizationQ5_K_S	Context32K	Est. memory≈ 7 GB	`ollama run mistrallite:7b-v0.1-q5_K_S`
7b-v0.1-q6_K	Typelocal	Inputstext	Digestc63d67539934	Download5.9 GB	Parameters7b	QuantizationQ6_K	Context32K	Est. memory≈ 8 GB	`ollama run mistrallite:7b-v0.1-q6_K`
7b-v0.1-q8_0	Typelocal	Inputstext	Digest8856fb347897	Download7.7 GB	Parameters7b	QuantizationQ8_0	Context32K	Est. memory≈ 10 GB	`ollama run mistrallite:7b-v0.1-q8_0`
latestdefault	Typelocal	Inputstext	Digest5393d4f5f262	Download4.1 GB	ParametersUnknown	QuantizationUnknown	Context32K	Est. memory≈ 6 GB	`ollama run mistrallite:latest`

Related models

sqlcoderlocal

SQLCoder is a code completion model fined-tuned on StarCoder for SQL generation tasks

openchatlocal

A family of open-source models trained on a wide variety of data, surpassing ChatGPT on various benchmarks. Updated to version 3.5-0106.

openhermeslocal

OpenHermes 2.5 is a 7B model fine-tuned by Teknium on Mistral with fully open datasets.

zephyrlocal

Zephyr is a series of fine-tuned versions of the Mistral and Mixtral models that are trained to act as helpful assistants.