Nbdev: A literate programming environment that democratizes software engineering best practices

2020-11-19 · Source: Hamel Husain's Blog · Field: Technology & Digital — Software Development & Engineering, Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, short

Summary

GitHub, in collaboration with fastai, has developed nbdev, a new literate programming environment for Python, released on November 20, 2020. This tool aims to democratize software development by integrating best practices like continuous integration, distribution, and documentation directly into Jupyter notebooks. nbdev automates the generation of searchable documentation hosted on GitHub Pages, sets up continuous integration with GitHub Actions for unit testing, and is fully compatible with GitHub Codespaces for easy setup. It also provides a two-way sync between notebooks and source code, allows in-notebook test writing, offers merge/conflict resolution for notebooks, and automates publishing to pypi and conda. This environment promotes higher quality software by removing friction from writing documentation and tests.

Key takeaway

For Python developers seeking to streamline their workflow and adhere to software engineering best practices, nbdev offers a compelling solution. You should consider adopting nbdev to integrate documentation, testing, and continuous integration directly within your Jupyter notebooks. This approach can significantly reduce friction in your development process, leading to more robust and well-documented software without needing to learn separate tools or APIs.

Key insights

nbdev integrates software engineering best practices directly into Jupyter notebooks, simplifying development workflows.

Principles

• Literate programming improves code clarity.
• Automated tooling lowers development barriers.
• Integrated documentation enhances software quality.

Method

nbdev builds on Jupyter notebooks, generating docs, setting up CI/CD, and enabling in-notebook testing and package publishing.

In practice

• Use nbdev for Python project development.
• Generate documentation automatically from notebooks.
• Automate package publishing to PyPI/Conda.

Topics

• nbdev
• Literate Programming
• Jupyter Notebooks
• Software Engineering Practices
• GitHub Integration

Code references

• features/codespaces
• features/packages
• fastai/nbdev
• features/actions
• fastai/fastpages

Best for: AI Engineer, NLP Engineer, Computer Vision Engineer, Software Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Why I Stopped Using nbdev — AI coding tools are shifting development paradigms, making idiosyncratic workflows like nbdev less efficient and collaborative.
• nbsanity - Share Notebooks as Polished Web Pages in Seconds — nbsanity offers instant, polished web rendering for public GitHub Jupyter notebooks using Quarto directives.
• কোডিং না জেনেই Lovable দিয়ে কীভাবে বানাবেন আপনার স্বপ্নের অ্যাপ? — Lovable.dev transforms natural language into full-stack web applications, democratizing app development for non-coders.
• 5 More Must-Know Python Concepts — Mastering advanced Python concepts like type hinting, functional tools, cooperative inheritance, pattern matching, and modern dependency management elevates code to professional standards.
• Reinventing the Python Notebook with Akshay Agrawal — Marimo reinvents Python notebooks with a reactive execution model, enabling reproducible, version-controlled, and interactive data exploration and application development.
• Why Notebooks Help You Understand What’s Really Happening — Interactive notebooks enhance human understanding and debugging across diverse application development contexts.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Hamel Husain's Blog.