Migrating from Neptune to Weights & Biases
Summary
Weights & Biases (W&B) is offering a migration guide and support for users transitioning from Neptune AI, which is shutting down on March 5th. The migration process involves exporting data from Neptune to local storage, then importing it into W&B. Users need W&B and Neptune API keys, and must clone the Neptune migration script repository. The guide details setting up environment variables for API keys, using UV for dependency management, and executing command-line scripts for data export and import. It also covers visualizing rich media, with an option to clone a separate repository for in-platform visualization. A key difference highlighted is W&B's project-centric run structure and its `wb.log` method for metrics, configurations, and media, contrasting with Neptune's `append` and `upload` functions. W&B also offers automatic dashboards and sharable saved views, differing from Neptune's manual comparison views.
Key takeaway
For MLOps Engineers and Data Scientists managing experiment tracking, you should prioritize exporting your Neptune AI data to local storage immediately, well before the March 5th shutdown. Leverage the provided migration scripts and W&B's support to transition your projects. Familiarize yourself with W&B's project-run structure and its unified `wb.log` method for metrics and media to ensure a smooth integration and avoid data loss.
Key insights
Migrate from Neptune AI to Weights & Biases by March 5th using provided scripts and a structured workflow.
Principles
- Prioritize data export before platform shutdown.
- Use environment variables for API key security.
- Understand platform-specific data logging paradigms.
Method
Clone the Neptune migration repo, install UV, set API keys as environment variables, export Neptune data using `neptune-exporter`, then import into W&B using `wandb-loader`, specifying entity and file paths.
In practice
- Export Neptune data locally immediately.
- Review Neptune projects to select data for migration.
- Use `wb.log` for all W&B metric and media logging.
Topics
- Neptune AI
- Weights & Biases
- Data Migration
- Experiment Tracking
- MLOps
Best for: Machine Learning Engineer, MLOps Engineer, Data Scientist
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Weights & Biases.