Build a free writing assistant with Python and Ollama

2024-02-27 · Source: Patrick Loeber · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, long

Summary

A tutorial demonstrates how to build a free, local typing assistant using Python and Ollama, inspired by a tweet on improving typing speed by correcting typos with AI. The application allows users to fix typos in selected text or the current line instantly via global hotkeys (F9 and F10). Key components include the `pi_input` library for keyboard control and hotkey listening, `pyperclip` for clipboard interaction, and Ollama for running a local large language model (LLM) like Mistral 7B. The system leverages Ollama's REST API to send text for correction and receive the processed output, integrating a prompt template to instruct the LLM to fix typos, casing, and punctuation while preserving newlines and returning only the corrected text. The entire setup is achieved with approximately 100 lines of Python code.

Key takeaway

For Python developers or AI enthusiasts looking to enhance productivity with local AI, consider building a custom typing assistant. This project demonstrates how to integrate `pi_input` for hotkeys and Ollama for local LLM inference, offering a practical way to automate text correction. You can adapt the prompt template and experiment with different Ollama models, like Mistral 7B, to optimize correction accuracy and speed for your specific needs.

Key insights

Build a local AI typing assistant using Python, Ollama, and keyboard/clipboard control for instant typo correction.

Principles

• Local LLMs enable free, private AI applications.
• Global hotkeys enhance user workflow efficiency.

Method

The method involves setting up `pi_input` for hotkey detection, `pyperclip` for clipboard management, and `httpx` to interact with a local Ollama LLM via its REST API. Text is copied, sent to the LLM with a specific prompt, and the corrected text is pasted back.

In practice

• Use `pi_input` for custom keyboard shortcuts.
• Integrate `pyperclip` for seamless clipboard operations.
• Deploy Ollama with quantized models for local performance.

Topics

• Local LLMs
• Ollama
• Python Automation
• Text Correction
• Prompt Engineering

Best for: Machine Learning Engineer, Software Engineer, AI Student

Related on AIssential

• Ollama: Unleash the Power of Local LLMs on Your Machine — Ollama enables private, cost-effective, and low-latency local LLM inference and customization via a Python library.
• OpenAI Codex with Ollama — OpenAI's Codex CLI now integrates with Ollama, enabling local execution of open-weight code generation models.
• Tool support — Ollama now enables LLMs to interact with external tools for complex tasks via a new API feature.
• 5 Cool Things I Did with Local Language Models — Local LLMs offer superior privacy, cost, and offline capabilities for sensitive data and proprietary workflows.
• From Local LLM to Tool-Using Agent — Local LLMs can become powerful tool-using agents by integrating with agent runtimes and external services.
• Build an LLM from Scratch 7: Instruction Finetuning — Instruction fine-tuning enables LLMs to follow diverse commands and generate free-form text responses.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Patrick Loeber.