datasette-llm-usage 0.2a0
Summary
The `datasette-llm-usage` plugin has been updated to version 0.2a0, focusing on tracking LLM token usage within a SQLite database. This release removes features related to allowances and estimated pricing, delegating these functions to the new `datasette-llm-accountant` plugin. It now relies on `datasette-llm` for model configuration. A significant new feature allows logging full prompts, responses, and tool calls to the `llm_usage_prompt_log` table in the internal database, enabled by the `datasette-llm-usage.log_prompts` plugin setting. Additionally, the `/ -/llm-usage-simple-prompt` page has been redesigned and now requires specific `llm-usage-simple-prompt` permission.
Key takeaway
For developers managing LLM deployments with Datasette, you should upgrade to `datasette-llm-usage 0.2a0` to streamline token tracking. Be aware that allowance and pricing features have moved to `datasette-llm-accountant`, requiring a separate installation for those capabilities. Consider enabling prompt logging for enhanced debugging and auditing of LLM interactions.
Key insights
The `datasette-llm-usage` plugin now focuses solely on LLM token usage tracking, offloading accounting features.
Principles
- Modularize LLM accounting functions
- Centralize LLM model configuration
Method
Configure `datasette-llm-usage.log_prompts` to enable detailed logging of LLM interactions (prompts, responses, tool calls) to `llm_usage_prompt_log` table.
In practice
- Install `datasette-llm-accountant` for pricing
- Set `log_prompts` to true for full logging
- Grant `llm-usage-simple-prompt` permission
Topics
- datasette-llm-usage
- LLM Token Tracking
- SQLite Database
- Prompt Logging
- datasette-llm
Best for: NLP Engineer, AI Engineer, Machine Learning Engineer, MLOps Engineer
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Simon Willison's Weblog.